Senior Computer Systems Engineer (Systems Infrastructure and Integration)

  • BART
  • Oakland, California
  • May 08, 2022
Full Time Engineering

Job Description


Ride BART to a satisfying career that lets you both: 1) make a difference to Bay Area residents, and 2) enjoy excellent pay, benefits and employment stability. BART is looking for people who like to be challenged, work in a fast paced environment, and have a passion for connecting riders to work, school and other places they need to go. BART offers a competitive salary, comprehensive health benefits, paid time off, and the CalPERS retirement program.

Conditions of Employment

All San Francisco Bay Area Rapid Transit District (BART) employees are required to be fully vaccinated against COVID-19 as a condition of employment. You will be required to show proof of your completed COVID-19 vaccination prior to receiving a final offer, unless you receive a COVID-19 Vaccination reasonable accommodation due to a medical condition or a religious exemption due to an approved sincerely held religious belief that prohibits you from receiving a vaccine. BART will review requests for a reasonable accommodation or religious exemption on a case-by-case basis. Documentation may be required. For questions, please contact BART Human Resources, Leave Management at HRDP@bart.gov .

Department

Computer Systems Engineering

Pay and Benefits

BART offers comprehensive compensation and benefits programs. Benefits include CalPERS pension; excellent medical (effective January 1, 2021 current employee cost $153.85 monthly for most plans), vision, and dental coverage; supplemental insurances; paid holidays and vacation; as well as two investment programs, one of which is entirely funded by BART. BART does not participate in Social Security. Complimentary BART passes for employee and qualifying dependents.

Pay Rate

Non Rep Pay Band 7
Minimum: $105,333.00, Midpoint: $131,667.00, Maximum: $159,580.00
Initial salary offer may be between $143,500.00 and $149,500.00 commensurate with experience

Posted Date

January 03, 2022

Closing Date

Open Until Filled / Further Notice
Initial Application Review will begin on February 3, 2022
Note that any job announcement may be closed/canceled at any time.

Reports To

Principal Computer Systems Engineer or designee

Days Off

Variable, As Assigned

Who May Apply

All current BART employees and qualified individuals who are not yet BART employees.

Current Assignment

The Systems and Data Analytics Engineering (SDAE) group at BART is looking for a systems infrastructure and integration engineer to join our team!
We are a group of over 30 engineers that develop, update, and support the Fare Collection and Rail Operations systems that are essential to BART and its riders. Our engineering experience span many disciplines including: embedded programming, mechanical control systems, electrical/mechanical engineering, public-facing electronic displays, maintenance-facing HMIs, Programmable Logic Controllers, our data center which runs VMware servers, KAFKA for applications integration, PCI compliant financial processing, cyber-security, and systems health monitoring and data analytics.
This assignment will be responsible for the systems infrastructure and integration underlying BART's revenue service for Fare Collection and Rail Operations. The successful candidate will help manage many 100d dozens of ES's of virtual machines anXI servers, 1000's of operationally critical endpoints, cyber-security, PCI compliance and guide BART in the deployment of tools to automate and standardize provisioning of production systems.
Primary Responsibilities:
General:
  • Work using Agile methodologies and Jira to coordinate work.
  • Mentor and teach colleagues.
  • Work with and manage vendors.
  • Develop, document, and implement processes to meet operational needs.
  • Support system audits and testing.
  • Ability to troubleshoot and problem solve.
  • Participate in production on-call rotation.
System Administration and System Engineering:
  • Manage Linux systems which are primarily Red Hat and Debian variants.
    • Write and maintain systems with scripts using Bash, Python and Perl.
  • Manage a variety of Windows systems.
    • Need to be familiar with Active Directory, SCCM and WSUS.
    • Write and maintain scripts using Powershell and Cmd.
  • Design, plan, deploy, document, and manage physical and virtual systems.
    • Need to be familiar with VMware ESXI's and vCenter Server.
    • Use tools like Ansible and Terraform to ensure systems are up to date and configured correctly.
    • Must be familiar with updating, upgrading, and replacing physical and software components in data center hardware.
  • Manage and ensure disaster recovery strategies and backups of critical systems are working.
    • Tools include Veritas Backup, Rsync and ssh.
  • Manage and deploy real-time monitoring and alert systems.
    • Tools include Nagios, Splunk, Kafka, Rsyslog, Auditd and SNMP agents.
    • Create dashboards that monitor infrastructure health and security concerns.
  • Proactively ensure high availability and acceptable levels of performance, security and capacity of mission critical resources.
  • Manage and maintain cyber security compliance across a variety of systems.
    • Updating and ensuring systems are secure when vulnerabilities are discovered.
    • Managing and deploying anti-virus, intrusion detection and proxy software.
    • Assist with PCI compliance and audits.
  • Work with engineering stakeholders to document requirements and design solutions.
  • Engineer solutions for projects.
Ideal incumbents would have the following beyond the minimum qualifications:
5+ years of experience in network operations, systems administration and computer and network security
Familiarity/proficiency with many of the following:
  • Hands on hardware troubleshooting experience.
  • PCI compliance best practices.
    • PCI QSA certified and up to date
  • Containers and OCI
  • Managing Apache Kafka clusters.
  • Utilizing DevOps practices such as CI/CD, infrastructure as code, and test automation using software like Jenkins, Gitlab pipelines and Nexus.
  • Linux administration.
  • Windows administration.
  • Authoring and maintaining technical documentation.
  • Handling and protection of financial or other sensitive data.
  • Centralized hardware and software monitoring and alerting.
  • Scope, schedule, and budget development.
  • Physical data center environments.
  • Advanced knowledge of Virtualized environments and operation.
  • Networking concepts: TCP/IP, routing, firewalls, NAT, VPN, VLAN, DNS.
  • Routers, security appliances, Layer 2 & 3 switches.
  • Network monitoring, threat detection/prevention, alerting and response
  • Centralized, reviewed, and planned operating system and 3rd party software upgrades
  • Strong analytical & problem-solving skills
  • Strong organizational skills
  • Strong written and verbal communication skills
  • Data analytics
  • Agile development and working with Scrum teams
  • Engineering: designing, prototyping and proof of concepts
  • Familiarity with contemporary high availability software architecture.
  • Software configuration management, change control, and version control systems such as GIT, coupled with a comprehensive bug-tracking system such as Bugzilla or Jira.
  • Experience in evaluating cost analyses and vendor comparisons for large scale projects and
  • Ability to work with a diverse team of engineers, security & compliance professionals and business stakeholders with various roles and responsibilities.

Essential Job Functions

Performs advanced computer systems engineering duties in control of the development, modification, installation and maintenance of the District's computerized equipment and systems; designs and tests computer control systems; monitors relative systems operations; provides direction to programmers and installation staff.
Resolves complex computer systems malfunctions; analyzes and determines cause, takes corrective actions; recommends system modifications as required.
Performs system database maintenance duties including analysis of space usage, backup, recovery and performance tuning recovery; reorganizes database configuration; maintains multiple database versions to support production and systems development operations; maintains and supports on line and off line databases in multiple locations.
Analyzes District plant engineering changes; determines requirements for database updates; analyzes impact on other subsystems.
Prepares plant for short and long term modification to existing computer systems; prepares cost benefit analysis and estimates for computer system projects.
Designs hardware and software modifications to control and data collection systems; prepares equipment specifications and procurement contracts.
Develops computer operating manuals and test procedures; prepares computer hardware and software installation, modification and testing.
Develops and modifies various complex programming codes and scripts; acts as liaison between the District and software consultants and equipment manufacturers.
Provides technical computer systems engineering support for various departments throughout the District in the maintenance of computer systems operations; analyzes user requirements; provides software or hardware solutions.
Prepares a variety of technical reports, manuals, specifications, drawings and correspondence relevant to project areas.
Attends and participates in professional group meetings; stays abreast of new trends and innovations in the field of computer systems engineering.
As assigned, may participate in training assigned employees in their areas of work including computer systems engineering design methods, procedures and techniques.

Minimum Qualifications

Education: Possession of a bachelor's degree in Computer Systems, Electronics, Electrical Engineering, or a closely related field from an accredited college or university.
Experience: The equivalent of three (3) years of full-time professional verifiable experience in computer systems engineering experience.
Substitution: Additional professional experience as outlined above may be substituted for the education on a year-for- year basis. A college degree is preferred.
Other Requirements: Must possess a valid California driver's license and have a satisfactory driving record. May be required to work overtime to test hardware and software.
Knowledge of:
Operations, services and activities of a comprehensive computer systems engineering program
Operational characteristics of computer systems and their sub-components
Computer programming languages and operating systems used in a variety of systems and real time applications
Methods and techniques of estimating and scheduling computer system modifications
Methods and techniques of programming and testing computer systems and applications
Operational characteristics of a variety of databases and operational systems
Methods and techniques of database management and administration
Computer hardware and software equipment and materials
Terminology, methods, practices, and techniques used in technical report preparation
Advanced mathematical principles
Current office procedures, methods, and equipment including computers
Principles of lead supervision and training
Related Federal, State and local laws, codes and regulations
Skill/ Ability in:
Developing, reviewing, and modifying complex computer systems
Leading, organizing and reviewing the work of lower level computer systems staff
Independently performing the most difficult computer systems engineering work
Reading and understanding technical drawings and specifications
Performing database management and administration duties
Interpreting and explaining District policies and procedures
Preparing complex computer hardware and software designs
Monitoring project budget and schedules
Preparing computer programs, test procedures and operating manuals
Troubleshooting and diagnosing computer system malfunctions
Preparing and interpreting computer hardware specifications
Analyzing complex technical problems, evaluating alternatives, and recommending solutions
Conducting software tests and debugging procedures
Understanding and following oral and written instructions
Communicating clearly and concisely, both orally and in writing
Establishing and maintaining effective working relationships with those contacted in the course of work including District officials and the general public

Selection Process

Applications will be screened to assure that minimum qualifications are met. Those applicants who meet minimum qualifications will then be referred to the hiring department for the completion of further selection processes.

The selection process for this position may include a skills/performance demonstration, a written examination, and a panel and/or individual interview.

The successful candidate must have an employment history demonstrating reliability and dependability; provide copies of certificates, diplomas or other documents as required by law, including those establishing his/her right to work in the U. S; pass a pre-employment medical examination which may include a drug and alcohol screen, and which is specific to the essential job functions and requirements. Pre-employment processing will also include a background check. (Does not apply to current full-time District employees unless specific job requires additional evaluations).

Application Process

External applicants may only apply online, at www.bart.gov/jobs. Applicants needing assistance with the online application process may receive additional information by calling (510) 464-6112.
Current employees are strongly encouraged to apply online, either at www.bart.gov/jobs, or on Employee Connect.
All applicants are asked to complete the application in full, indicating dates of employment, all positions held, hours worked, and a full description of duties. Online applicants are invited to electronically attach a resume to the application form to provide supplemental information but should not consider the resume a substitute for the application form itself.
Applications must be complete by the closing date and time listed on the job announcement.
When you have successfully applied for this position, you will receive an auto reply e-mail acknowledging that your application was received for this position. Please retain a copy of the e-mail for your records. If you receive an auto reply that does not specifically reference this position, please email Employment Help at employment@bart.gov for assistance.
To verify submission of your application, click on the 'My Career Tools' link at the top of the 'Careers Home Page' after submitting your application to view the list of applications you have submitted (including application date and status). If you have further questions, please email the Employment Help at employment@bart.gov, between the hours of 8:15am - 5:00pm, Monday- Friday.

Equal Employment Opportunity

The San Francisco Bay Area Rapid Transit District is an equal opportunity employer. Applicants shall not be discriminated against because of race, color, sex, sexual orientation, gender identity, gender expression, age (40 and above), religion, national origin (including language use restrictions), disability (mental and physical, including HIV and AIDS), ancestry, marital status, military status, veteran status, medical condition (cancer/genetic characteristics and information), or any protected category prohibited by local, state or federal laws.

The BART Human Resources Department will make reasonable efforts in the examination process to accommodate persons with disabilities or for religious reasons. Please advise the Human Resources Department of any special needs in advance of the examination by emailing at least 5 days before your examination date at employment@bart.gov .

Qualified veterans may be eligible to obtain additional veteran's credit in the selection process for this recruitment (effective Jan. 1, 2013). To obtain the credit, veterans must attach to the application a DD214 discharge document or proof of disability and complete/submit the Veteran's Preference Application no later than the closing date of the posting. For more information about this credit please go to the Veteran's Preference Policy and Application link at www.bart.gov/jobs.


Other Information

Working Conditions:
Environmental Conditions: Office environment; exposure to computer screens; field environment; travel from site to site; construction site environment; exposure to noise, dust, grease, smoke, fumes, gases, heat, cold, and inclement weather conditions when conducting field inspections and investigations.
Physical Conditions: Requires maintaining physical condition necessary for walking, standing or sitting for prolonged periods of time.



Essential Job Functions

The Systems and Data Analytics Engineering (SDAE) group at BART is looking for a systems infrastructure and integration engineer to join our team!
We are a group of over 30 engineers that develop, update, and support the Fare Collection and Rail Operations systems that are essential to BART and its riders. Our engineering experience span many disciplines including: embedded programming, mechanical control systems, electrical/mechanical engineering, public-facing electronic displays, maintenance-facing HMIs, Programmable Logic Controllers, our data center which runs VMware servers, KAFKA for applications integration, PCI compliant financial processing, cyber-security, and systems health monitoring and data analytics.
This assignment will be responsible for the systems infrastructure and integration underlying BART's revenue service for Fare Collection and Rail Operations. The successful candidate will help manage many 100d dozens of ES's of virtual machines anXI servers, 1000's of operationally critical endpoints, cyber-security, PCI compliance and guide BART in the deployment of tools to automate and standardize provisioning of production systems.
Primary Responsibilities:
General:
  • Work using Agile methodologies and Jira to coordinate work.
  • Mentor and teach colleagues.
  • Work with and manage vendors.
  • Develop, document, and implement processes to meet operational needs.
  • Support system audits and testing.
  • Ability to troubleshoot and problem solve.
  • Participate in production on-call rotation.
System Administration and System Engineering:
  • Manage Linux systems which are primarily Red Hat and Debian variants.
    • Write and maintain systems with scripts using Bash, Python and Perl.
  • Manage a variety of Windows systems.
    • Need to be familiar with Active Directory, SCCM and WSUS.
    • Write and maintain scripts using Powershell and Cmd.
  • Design, plan, deploy, document, and manage physical and virtual systems.
    • Need to be familiar with VMware ESXI's and vCenter Server.
    • Use tools like Ansible and Terraform to ensure systems are up to date and configured correctly.
    • Must be familiar with updating, upgrading, and replacing physical and software components in data center hardware.
  • Manage and ensure disaster recovery strategies and backups of critical systems are working.
    • Tools include Veritas Backup, Rsync and ssh.
  • Manage and deploy real-time monitoring and alert systems.
    • Tools include Nagios, Splunk, Kafka, Rsyslog, Auditd and SNMP agents.
    • Create dashboards that monitor infrastructure health and security concerns.
  • Proactively ensure high availability and acceptable levels of performance, security and capacity of mission critical resources.
  • Manage and maintain cyber security compliance across a variety of systems.
    • Updating and ensuring systems are secure when vulnerabilities are discovered.
    • Managing and deploying anti-virus, intrusion detection and proxy software.
    • Assist with PCI compliance and audits.
  • Work with engineering stakeholders to document requirements and design solutions.
  • Engineer solutions for projects.
Ideal incumbents would have the following beyond the minimum qualifications:
5+ years of experience in network operations, systems administration and computer and network security
Familiarity/proficiency with many of the following:
  • Hands on hardware troubleshooting experience.
  • PCI compliance best practices.
    • PCI QSA certified and up to date
  • Containers and OCI
  • Managing Apache Kafka clusters.
  • Utilizing DevOps practices such as CI/CD, infrastructure as code, and test automation using software like Jenkins, Gitlab pipelines and Nexus.
  • Linux administration.
  • Windows administration.
  • Authoring and maintaining technical documentation.
  • Handling and protection of financial or other sensitive data.
  • Centralized hardware and software monitoring and alerting.
  • Scope, schedule, and budget development.
  • Physical data center environments.
  • Advanced knowledge of Virtualized environments and operation.
  • Networking concepts: TCP/IP, routing, firewalls, NAT, VPN, VLAN, DNS.
  • Routers, security appliances, Layer 2 & 3 switches.
  • Network monitoring, threat detection/prevention, alerting and response
  • Centralized, reviewed, and planned operating system and 3rd party software upgrades
  • Strong analytical & problem-solving skills
  • Strong organizational skills
  • Strong written and verbal communication skills
  • Data analytics
  • Agile development and working with Scrum teams
  • Engineering: designing, prototyping and proof of concepts
  • Familiarity with contemporary high availability software architecture.
  • Software configuration management, change control, and version control systems such as GIT, coupled with a comprehensive bug-tracking system such as Bugzilla or Jira.
  • Experience in evaluating cost analyses and vendor comparisons for large scale projects and
  • Ability to work with a diverse team of engineers, security & compliance professionals and business stakeholders with various roles and responsibilities.

Essential Job Functions

Performs advanced computer systems engineering duties in control of the development, modification, installation and maintenance of the District's computerized equipment and systems; designs and tests computer control systems; monitors relative systems operations; provides direction to programmers and installation staff.
Resolves complex computer systems malfunctions; analyzes and determines cause, takes corrective actions; recommends system modifications as required.
Performs system database maintenance duties including analysis of space usage, backup, recovery and performance tuning recovery; reorganizes database configuration; maintains multiple database versions to support production and systems development operations; maintains and supports on line and off line databases in multiple locations.
Analyzes District plant engineering changes; determines requirements for database updates; analyzes impact on other subsystems.
Prepares plant for short and long term modification to existing computer systems; prepares cost benefit analysis and estimates for computer system projects.
Designs hardware and software modifications to control and data collection systems; prepares equipment specifications and procurement contracts.
Develops computer operating manuals and test procedures; prepares computer hardware and software installation, modification and testing.
Develops and modifies various complex programming codes and scripts; acts as liaison between the District and software consultants and equipment manufacturers.
Provides technical computer systems engineering support for various departments throughout the District in the maintenance of computer systems operations; analyzes user requirements; provides software or hardware solutions.
Prepares a variety of technical reports, manuals, specifications, drawings and correspondence relevant to project areas.
Attends and participates in professional group meetings; stays abreast of new trends and innovations in the field of computer systems engineering.
As assigned, may participate in training assigned employees in their areas of work including computer systems engineering design methods, procedures and techniques.

Minimum Qualifications

Education: Possession of a bachelor's degree in Computer Systems, Electronics, Electrical Engineering, or a closely related field from an accredited college or university.
Experience: The equivalent of three (3) years of full-time professional verifiable experience in computer systems engineering experience.
Substitution: Additional professional experience as outlined above may be substituted for the education on a year-for- year basis. A college degree is preferred.
Other Requirements: Must possess a valid California driver's license and have a satisfactory driving record. May be required to work overtime to test hardware and software.
Knowledge of:
Operations, services and activities of a comprehensive computer systems engineering program
Operational characteristics of computer systems and their sub-components
Computer programming languages and operating systems used in a variety of systems and real time applications
Methods and techniques of estimating and scheduling computer system modifications
Methods and techniques of programming and testing computer systems and applications
Operational characteristics of a variety of databases and operational systems
Methods and techniques of database management and administration
Computer hardware and software equipment and materials
Terminology, methods, practices, and techniques used in technical report preparation
Advanced mathematical principles
Current office procedures, methods, and equipment including computers
Principles of lead supervision and training
Related Federal, State and local laws, codes and regulations
Skill/ Ability in:
Developing, reviewing, and modifying complex computer systems
Leading, organizing and reviewing the work of lower level computer systems staff
Independently performing the most difficult computer systems engineering work
Reading and understanding technical drawings and specifications
Performing database management and administration duties
Interpreting and explaining District policies and procedures
Preparing complex computer hardware and software designs
Monitoring project budget and schedules
Preparing computer programs, test procedures and operating manuals
Troubleshooting and diagnosing computer system malfunctions
Preparing and interpreting computer hardware specifications
Analyzing complex technical problems, evaluating alternatives, and recommending solutions
Conducting software tests and debugging procedures
Understanding and following oral and written instructions
Communicating clearly and concisely, both orally and in writing
Establishing and maintaining effective working relationships with those contacted in the course of work including District officials and the general public

Selection Process

Applications will be screened to assure that minimum qualifications are met. Those applicants who meet minimum qualifications will then be referred to the hiring department for the completion of further selection processes.

The selection process for this position may include a skills/performance demonstration, a written examination, and a panel and/or individual interview.

The successful candidate must have an employment history demonstrating reliability and dependability; provide copies of certificates, diplomas or other documents as required by law, including those establishing his/her right to work in the U. S; pass a pre-employment medical examination which may include a drug and alcohol screen, and which is specific to the essential job functions and requirements. Pre-employment processing will also include a background check. (Does not apply to current full-time District employees unless specific job requires additional evaluations).

Application Process

External applicants may only apply online, at www.bart.gov/jobs. Applicants needing assistance with the online application process may receive additional information by calling (510) 464-6112.
Current employees are strongly encouraged to apply online, either at www.bart.gov/jobs, or on Employee Connect.
All applicants are asked to complete the application in full, indicating dates of employment, all positions held, hours worked, and a full description of duties. Online applicants are invited to electronically attach a resume to the application form to provide supplemental information but should not consider the resume a substitute for the application form itself.
Applications must be complete by the closing date and time listed on the job announcement.
When you have successfully applied for this position, you will receive an auto reply e-mail acknowledging that your application was received for this position. Please retain a copy of the e-mail for your records. If you receive an auto reply that does not specifically reference this position, please email Employment Help at employment@bart.gov for assistance.
To verify submission of your application, click on the 'My Career Tools' link at the top of the 'Careers Home Page' after submitting your application to view the list of applications you have submitted (including application date and status). If you have further questions, please email the Employment Help at employment@bart.gov, between the hours of 8:15am - 5:00pm, Monday- Friday.

Equal Employment Opportunity

The San Francisco Bay Area Rapid Transit District is an equal opportunity employer. Applicants shall not be discriminated against because of race, color, sex, sexual orientation, gender identity, gender expression, age (40 and above), religion, national origin (including language use restrictions), disability (mental and physical, including HIV and AIDS), ancestry, marital status, military status, veteran status, medical condition (cancer/genetic characteristics and information), or any protected category prohibited by local, state or federal laws.

The BART Human Resources Department will make reasonable efforts in the examination process to accommodate persons with disabilities or for religious reasons. Please advise the Human Resources Department of any special needs in advance of the examination by emailing at least 5 days before your examination date at employment@bart.gov .

Qualified veterans may be eligible to obtain additional veteran's credit in the selection process for this recruitment (effective Jan. 1, 2013). To obtain the credit, veterans must attach to the application a DD214 discharge document or proof of disability and complete/submit the Veteran's Preference Application no later than the closing date of the posting. For more information about this credit please go to the Veteran's Preference Policy and Application link at www.bart.gov/jobs.


Other Information

Working Conditions:
Environmental Conditions: Office environment; exposure to computer screens; field environment; travel from site to site; construction site environment; exposure to noise, dust, grease, smoke, fumes, gases, heat, cold, and inclement weather conditions when conducting field inspections and investigations.
Physical Conditions: Requires maintaining physical condition necessary for walking, standing or sitting for prolonged periods of time.

Job Address

Oakland, California United States View Map