Linux System Administrator (CSP 2 / 3) | Los Alamos, NM | Los Alamos National Laboratory

Linux System Administrator (CSP 2 / 3)

What You Will Do

The High Performance Computing (HPC) Division at Los Alamos National Laboratory provides scientific computing resources consisting of some of the largest HPC systems in the world, including a large (19K+ node) Cray system called Trinity, as well as numerous large commodity cluster systems. Our High Performance Computing (HPC) Computer System Professional (CSP) Team within the HPC Systems Group (HPC-SYS) provides vanguard production monitoring, support, testing, and maintenance for existing systems and deployment support for future systems. Visit the HPC website to learn more: https://www.lanl.gov/org/ddste/aldsc/hpc/index.php

The CSP Team is seeking our next dynamic team member to help deploy and maintain our existing and future HPC systems. Mentoring of students, junior staff, and peers in technical and professional growth activities is highly valued, as is maintaining state-of-the-art technical expertise and knowledge within HPC and developing new skills in related disciplines. This position will be filled at either the Computing Systems Professional 2 or Computing Systems Professional 3 level, depending on the skills of the selected candidate. Additional job responsibilities (outlined below) will be assigned if the candidate is hired at the higher level.

Computing Systems Professional 2 $77,300 - $126,000

The successful Computing System Professional 2 candidate will participate in periodic on-call responsibilities and actively grow HPC skill base and expertise across networking, data storage, system administration as part of the HPC-SYS Triage Team. Specific tasks/scenarios in which the selected candidate will engage in include; deploying and testing new hardware, troubleshooting and diagnosing system failures, and modifying existing systems, software and methods while actively participating in knowledge sharing across teams.

Computing Systems Professional 3 $94,100 - $155,700

In addition to what was outlined at the lower level, at this level A successful Computing System Professional 3 candidate will participate in periodic on-call responsibilities and apply subject matter expertise in one or more core topical areas (system, network, or data storage administration), both independently and collaboratively with other members of the team or group, after receiving initial direction and requirements from technical project leads. In addition, the selected candidate will actively grow HPC skill base and expertise across networking, data storage, system administration as part of the HPC-SYS Triage Team. Specific tasks/scenarios in which the selected candidate will engage in include: deploying and testing new hardware, troubleshooting and diagnosing system failures, and modifying existing systems, software and methods while actively participating in knowledge sharing across teams. In addition, the selected candidate will have the opportunity to develop technical products such as technical documentation, presentations, technical papers, and reports, to communicate findings internally.

What You Need
Minimum Job Requirements:



Computing System Professional 2
  • Demonstrated knowledge of building, configuring, troubleshooting, and administering Linux computer/support systems, including Linux command line interface skills, and experience scripting in Bash, Perl, Python, or similar languages.
  • Demonstrated effective communication skills, including demonstrated ability to work productively with customers and suppliers.
  • Demonstrated ability to work in a team environment.
  • Ability to obtain a Q clearance, which typically requires U.S. citizenship.
  • Proven track record of continuous learning to advance technical skillsets and knowledge.

Additional Job Requirements for Computing System Professional 3:



Computing System Professional 3


In addition to the Job Requirements outlined above, qualification at the CSP-3 level requires:

• Proven ability to work independently and in a team environment to analyze problems, propose solutions to management, and deploy and document implemented solutions.

• Demonstrated experience building, configuring, and administering production Linux computer/support systems, including strong command line Linux operating system skills, working knowledge of or experience with hardware and software security practices, and intermediate experience scripting in Bash, Perl, Python, or similar languages.

• Demonstrated experience in automating tasks using programming and scripting

• Ability to program in a compiled or interpretative language

• Broad experience in network administration, including knowledge of TCP/IP, Ethernet, and/or High-Speed Networks (such as InfiniBand or Omni-Path) and/or Broad experience in data storage administration, including knowledge of storage system hardware.

• Experience communicating technical information to both technical and non-technical personnel

• Demonstrated ability to communicate technical strategy, accomplishments, and challenges to management team, as well as cross-organizationally.

Education/Experience at lower level: CSP-2 Position requires a bachelor's degree from an accredited college or university and a minimum of four years of related experience, or an equivalent combination of education and experience. At this level, applicable advanced vendor and/or professional certification is desirable.

Education/Experience at higher level: CSP-3 Position requires a bachelor's degree from an accredited college or university and a minimum of eight years of related experience, or an equivalent combination of education and experience. At this level, applicable advanced vendor and/or professional certification is desirable.

Desired Qualifications:



• Experience working in a production computing environment, preferably with HPC data centers, large topology systems or at large scale.

• Experience supporting a scientific user base and/or experience managing computers in a DOE or DOD classified environment.

• Demonstrated experience with centralized configuration management in a heterogeneous computing environment.

• Demonstrated experience working with authentication services such as LDAP

• Demonstrated experience maintaining various system services (Kerberos, NFS, SSH, Samba, etc.)

• Experience integrating operational metrics into a monitoring system such as Splunk.

• Experience configuring networks, network switches, firewalls. Experience with multiple network technologies (e.g., Ethernet, IB, OPA).

• Experience with multiple Linux distributions; experience diagnosing system software problems; familiarity with Cfengine, Chef, Puppet, Ansible, Salt, or similar configuration and automation tools and practices; experience with revision control systems such as RCS, Subversion, or Git; and/or experience with low-level system administration tools such as perf, strace, tcpdump, and vmstat.

• Knowledge of parallel/distributed file systems (e.g., Lustre, GPFS, Panasas, Glustre).

• Knowledge of file systems such as ZFS, EXT, XFS and their underlying structures/characteristics.

• Experience with Object storage and RESTful storage interfaces.

• Experience with archival storage systems.

• Basic understanding of relational databases and database design methodologies.

• Contribution to open source or non-work-related projects.

• Demonstrated experience leading and mentoring teams, students, or junior team members.

• An Active DOE Q Clearance.

• An Active SCI Clearance.

Location: This position will be located in Los Alamos, NM.

COVID Vaccine:

The COVID vaccine is mandatory for all Laboratory employees, on-site contractors, and on-site subcontractors unless granted an accommodation under applicable state or federal law. This requirement will apply to those working on-site, those teleworking, and all new hires.

Position commitment: Regular appointment employees are required to serve a period of continuous service in their current position in order to be eligible to apply for posted jobs throughout the Laboratory. If an employee has not served the time required, they may only apply for Laboratory jobs with the documented approval of their Division Leader. The position commitment for this position is 1 year.

Note to Applicants: For consideration, applicants should submit a cover letter addressing how their knowledge, skills and abilities meet the minimum requirements along with a resume.
Where You Will Work



Located in beautiful northern New Mexico, Los Alamos National Laboratory (LANL) is a multidisciplinary research institution engaged in strategic science on behalf of national security. Our generous benefits package includes:

§ PPO or High Deductible medical insurance with the same large nationwide network

§ Dental and vision insurance

§ Free basic life and disability insurance

§ Paid childbirth and parental leave

§ Award-winning 401(k) (6% matching plus 3.5% annually)

§ Learning opportunities and tuition assistance

§ Flexible schedules and time off (paid sick, vacation, and holidays)

§ Onsite gyms and wellness programs

§ Extensive relocation packages (outside a 50 mile radius)
Additional Details

Directive 206.2 - Employment with Triad requires a favorable decision by NNSA indicating employee is suitable under NNSA Supplemental Directive 206.2. Please note that this requirement applies only to citizens of the United States. Foreign nationals are subject to a similar requirement under DOE Order 142.3A.

Clearance: Q (Position will be cleared to this level). Applicants selected will be subject to a Federal background investigation and must meet eligibility requirements* for access to classified matter. This position requires a Q clearance which requires US Citizenship except in extremely rare circumstances. Dependent upon position, additional authorization to access nuclear weapons information may be required that may or may not be available to dual citizens depending upon the circumstances.

*Eligibility requirements: To obtain a clearance, an individual must be at least 18 years of age; U.S. citizenship is required except in very limited circumstances. See DOE Order 472.2 for additional information.

New-Employment Drug Test: The Laboratory requires successful applicants to complete a new-employment drug test and maintains a substance abuse policy that includes random drug testing.

Regular position: Term status Laboratory employees applying for regular-status positions are converted to regular status.

Internal Applicants: Regular appointment employees who have served the required period of continuous service in their current position are eligible to apply for posted jobs throughout the Laboratory. If an employee has not served the required period of continuous service, they may only apply for Laboratory jobs with the documented approval of their Division Leader. Please refer to Policy Policy P701 for applicant eligibility requirements.
Equal Opportunity: Los Alamos National Laboratory is an equal opportunity employer and supports a diverse and inclusive workforce. All employment practices are based on qualification and merit, without regard to race, color, national origin, ancestry, religion, age, sex, gender identity, sexual orientation or preference, marital status or spousal affiliation, physical or mental disability, medical conditions, pregnancy, status as a protected veteran, genetic information, or citizenship within the limits imposed by federal laws and regulations. The Laboratory is also committed to making our workplace accessible to individuals with disabilities and will provide reasonable accommodations, upon request, for individuals to participate in the application and hiring process. To request such an accommodation, please send an email to applyhelp@lanl.gov or call 1-505-665-4444 option 1.Employment StatusFull Time