Job Description
Senior Database Engineer (Elastic/Mongo/Hadoop)
Join to apply for the Senior Database Engineer (Elastic/Mongo/Hadoop) role at CatchProbe Intelligence Technologies
Workplace Type: Remote
Region: San Francisco, CA
Job Description
- Must have experience with MongoDB installations, upgrades, support on MongoDB
- Responsible for administration, maintenance, performance analysis, and capacity planning for MongoDB/Elastic/Hadoop clusters.
- Coordinate and plan with application teams on MongoDB capacity planning for new applications.
- Should have knowledge of using mongo base tools like mongodump, mongoexport, mongorestore, mongoimport, mongostat, mongotop
- Must be well-versed with JSON scripting, writing queries in mongo in shell scripts and in the mongo shell
- Should be able to support sharded clusters and perform upgrades and other config maintenance on sharded clusters
- Must be able to address, monitor and manage capacity requirements all aspects CPU, memory and storage.
- Must be able to assist application teams with assessment and/or resolution of performance bottlenecks observed in the MongoDB Database tier of their stack
- Must be aware of different authentication/authorization methods used in Mongo SCRAM-SHA1, X509, LDAP including reconfiguration of instances with TLS/SSL and/or LDAP
- Candidate must also be able to develop automated solutions for ad-hoc script execution requests, ad-hoc report generation, upgrades, installs
- Experienced in NoSQL DB technologies
- Must be aware of how to use Cloud manager and share relevant metrics for a given deployment when an issue arises
- Must have experience with Docker, deploying Mongo containers running on Docker and supporting all aspects of MongoDB administration needs within a Docker container
- Knowledge of administration and support of Hadoop systems will be an added advantage
- Deploy Hadoop (Bigdata) cluster, comm/decommissioning of nodes, track jobs, monitor services like ZooKeeper, HBase, SOLR indexing, configure name-node HA, schedule, configuring backups & restore
- Develop script to review logs and alert in case of long running queries
- Demonstrable expertise in deployment and use of Postgres/MySQL, Kafka/Kinesis, etc.
- Strong scripting experience with Python (preferred), and Shell (secondary)
- Individually build services, and expose internal APIs for these services that allow other teams and workflows to use data infrastructure automation components.
Required Skills
- Strong understanding of various relational and non-relational database technologies and their benefits, downsides and best use-case and help application teams to use the correct database technology based on their specific business use case
- 5+ years installing, automating, scaling and supporting NoSQL databases such as MongoDB, ElasticSearch, Hadoop, among other emerging technologies
- 1-2 years experience working with the databases in public clouds like Hetzner, AWS, Azure and GCP
- Proficiency in automation
- Knowledge of Ansible, Python, Terraform
- Willingness and commitment to learn other database, automation, and cloud technologies
- Great communication and collaboration skills
- Software development experience and knowledge of modern software development processes
- Knowledge/experience in development programming languages such as Java, Go, Node.js, HTML, CSS, Bootstrap etc. is a plus
- Knowledge/experience on AI/ML is a big plus
- Ability to multi-task and prioritize with little to no supervision, providing team leadership skills
- Ability to work well under pressure
- Consistent exercise of independent judgement and discretion in matters of significance
- Excellent communication skills
- Highly driven, highly involved, highly proactive
- Datalake cluster ownership and technical point of contact for all applications on Hadoop cluster
- Responsible for new application onboarding in Datalake by reviewing requirement and design
- Assist existing and new applications to come up with most optimized and suitable solutions for their requirement
- L3 point of contact for issues related to Hadoop platform
Core Responsibilities
- Develop solutions for very complex and wide-reaching systems engineering problems, set new policies and procedures, create systems engineering and architectural documentation
- Operating systems & disk management: provide in-depth knowledge, mentor junior team members, create basic task automation scripts
- Database platform management: master understanding of database concepts, availability, performance, usage and configuration, set up, fix and tune complex replication
- Storage and backup: set up, solve problems and tune complex SAN software issues, maintain policies and documentation
- Scripting and development: develop software in several modern languages, design horizontally-scalable solutions and apply professional standards
- Networking: recommend or help architect an entire system, perform network sniffing, understand protocols
- Application technologies: provide recommendations and advice regarding web services, OS and storage, liaise with development, QA and business teams
- Analyze systems, make recommendations to prevent problems, lead issue resolution activities
- Lead end-to-end audit of monitors and alarms, define requirements for new tools
- Apply time and project management skills to lead resolution of issues, communicate necessary information, consult with clients or third-party vendors
- Consistent exercise of independent judgement and discretion in matters of significance
- Regular, consistent and punctual attendance
Seniority level: Mid-Senior level
Employment type: Full-time
Job function: Information Technology
Industries: Software Development
Referrals increase your chances of interviewing at CatchProbe Intelligence Technologies by 2x
#J-18808-Ljbffr
Job Tags
Full time, Remote work,