Senior Database Engineer(Elastic/Mongo/Hadoop)

Company: CatchProbe Intelligence Technologies
Location: San Francisco
Posted on: May 3, 2025

Job Description:

Senior Database Engineer (Elastic/Mongo/Hadoop)Senior Database Engineer (Elastic/Mongo/Hadoop)Workplace Type: Remote - Region: San Francisco, CAJob Description

• Must have experience with MongoDB installations, upgrades, support on MongoDB.
• Responsible for administration, maintenance, performance analysis, and capacity planning for MongoDB/Elastic/Hadoop clusters.
• Coordinate and plan with application teams on MongoDB capacity planning for new applications.
• Should have knowledge of using MongoDB tools - like mongodump, mongoexport, mongorestore, mongoimport, mongostat, mongotop.
• Must be well-versed with JSON scripting, writing queries in MongoDB shell scripts and in the mongo shell.
• Should be able to support sharded clusters and perform upgrades and other configuration maintenance on sharded clusters.
• Must be able to address, monitor and manage capacity requirements - all aspects CPU, memory and storage.
• Must be able to assist application teams with assessment and/or resolution of performance bottlenecks observed in the MongoDB Database tier of their stack.
• Must be aware of different authentication/authorization methods used in MongoDB - SCRAM-SHA1, X509, LDAP - including reconfiguration of instances with TLS/SSL and/or LDAP.
• Candidate must also be able to develop automated solutions for ad-hoc script execution requests, ad-hoc report generation, upgrades, installs.
• Experienced in NoSQL DB technologies.
• Must be aware of how to use Cloud Manager and share relevant metrics for a given deployment when an issue arises.
• Must have experience with Docker, deploying Mongo containers running on Docker and supporting all aspects of MongoDB administration needs within a Docker container.
• Knowledge of administration and support of Hadoop systems will be an added advantage.
• Deploy Hadoop (Big Data) cluster, comm/decommissioning of nodes, track jobs, monitor services like Zookeeper, HBase, SOLR indexing, configure name-node HA, schedule, configuring backups & restore.
• To develop scripts to review logs and alert us in case of long-running queries.
• Demonstrable expertise (deployment, use) in some of the following data storage technologies: Postgres/MySQL ("on-prem", RDS, Aurora), Kafka/Kinesis ("on-prem" or managed).
• Strong scripting experience with Python (preferred), and Shell (secondary).
• Individually build services, and expose internal APIs for these services that allow other teams and workflows to use data infrastructure automation components.Required Skills
• Strong understanding of various relational and non-relational database technologies along with their benefits, downsides and best use-case and help application teams to use the correct database technology based on their specific business use case.
• 5+ years installing, automating, scaling and supporting NoSQL databases such as MongoDB, ElasticSearch, Hadoop, among other emerging technologies.
• 1-2 years experience working with the databases in public clouds like Hetzner, AWS, Azure and GCP.
• Proficiency in automation.
• Knowledge of Ansible, Python, Terraform.
• Willingness and commitment to learn other database, automation, and cloud technologies.
• Great communication and collaboration skills.
• Software development experience and knowledge of modern software development processes.
• Knowledge/experience in development programming languages such as Java, Go, NodeJS, HTML, CSS, Bootstrap etc. is a plus.
• Knowledge/experience on AI/ML is a big plus.
• Ability to multi-task and prioritize with little to no supervision, providing team leadership skills.
• Ability to work well under pressure.
• Consistent exercise of independent judgment and discretion in matters of significance.
• Excellent communication skills.
• Highly driven, highly involved, highly proactive.
• Datalake cluster ownership and technical point of contact for all applications on Hadoop cluster.
• Responsible for new application onboarding in Datalake by reviewing requirements and design.
• Assist existing and new applications to come up with the most optimized and suitable solutions for their requirements.
• L3 point of contact for issues related to the Hadoop platform.Core Responsibilities
• Develop solutions for very complex and wide-reaching systems engineering problems. Set new policies and procedures to handle future issues. Create systems engineering and architectural documentation to be used by others to build and maintain systems.
• Operating Systems & Disk Management: Provide in-depth knowledge of Operating System internals to aid in solving complex problems. Act as an authority on at least one supported Operating System. Mentor and train more junior team members on Operating System concepts, configuration, tuning and troubleshooting techniques. Create basic task automation scripts in Shell, Perl, VBScript, or similar. May manage servers remotely in a distributed environment.
• Database Platform Management: Master understanding of database concepts, availability, performance, usage and configuration. Set up, fix and tune complex standard and non-standard replication. Use knowledge of existing database platforms to evaluate and recommend new technologies. Use database knowledge to solve issues on unfamiliar products. Create and maintain database policies, standards and overall documentation including availability, replication, availability and backup and recovery policy, service level agreement, baseline architecture, change management, access to production, unsupported HW/SW, security & audit violations and risk acceptance.
• Storage and Backup: Set up, solve problems and tune complex SAN software issues. Use knowledge of existing storage platforms to evaluate and recommend new technologies. Use storage knowledge to solve issues on unfamiliar products. Create and maintain policies, standards and overall documentation including availability and backup & recovery, service level agreement, baseline architecture, change management, access to production, unsupported HW/SW, security & audit violations and risk acceptance.
• Scripting and Development: Develop software in several modern languages. Develop database-backed systems and have a solid grasp of DB schema and query performance. Design horizontally-scalable solutions with innovative use of storage and networking, including solid APIs for integration with other systems. Apply professional standard methodologies in day-to-day work like revision control, unit testing, or other. Apply statistical data analysis techniques.
• Networking: Recommend or help architect an entire system, including network design and topology. Authority in understanding and performing TCP dumps, snoop and other network sniffers. Understand and apply knowledge of most protocols (TCP/IP, HTTP, UDP etc.).
• Application Technologies (Web Servers, J2EE, Applications Servers): Provide authoritative recommendations and advice to the team and/or department in the areas of web services, OS and storage, including being an active liaison to Development, QA and the Business. Provide scaling, design, costing, troubleshooting and impact analysis consultation.
• Analyze systems and make recommendations to prevent possible problems. Take lead on issue resolution activities using knowledge of complex and company-wide systems.
• Lead end-to-end audit of monitors and alarms based on subsystem knowledge. Take the lead on defining the requirements for new tools required for operations.
• Apply time management and project management skills to lead the resolution of issues in a timely and organized manner, effectively communicating necessary information. May consult directly with clients or third-party vendors; provide domain expertise.
• Consistent exercise of independent judgment and discretion in matters of significance.
• Regular, consistent and punctual attendance.
  #J-18808-Ljbffr

Keywords: CatchProbe Intelligence Technologies, Stockton , Senior Database Engineer(Elastic/Mongo/Hadoop), IT / Software / Systems , San Francisco, California