Post to Multiple Job Boards



Now take benefits of multiple job boards and search engine posting.
Try it.



Job Datalake/Bigdata AWS Resource Massachusetts Datalake/Bigdata

Datalake/Bigdata AWS Resource

Skills:Datalake/Bigdata       |  Location:Massachusetts  ,  United States Of America

Views:10


Job Title : Datalake/Bigdata AWS Resource
Location : Cambridge (Boston), MA
Project : 6 Months+

Mandatory Skills :: Big Data, Cloud and Excellent Communication skill

Job Description:

Datalake/Hadoop Resource

·         Implementation and Administration of On-prem Data lake environment

·         Monitoring and managing the Hadoop services on 3 clusters

·         Installing the New hosts (Head nodes, compute nodes and worker
nodes to the existing cluster) and decommission of the hosts from the cluster

·         Maintenance and Monitoring of the jobs of  Production, UAT and
Development environments

·         Code changes and updated code deployments in the UAT and Production
environments

·         Deploying code changes on Rshiny server  and Rstudio server as per
the user request

·         Implementation and Monitoring of oozie scheduled jobs

·         Implementation of patching activities and applying the fixes to the
data lake environment provided by the Hortonworks

·         Working on the job failures mostly Hive and Spark jobs across the
data lake environment

·         Onboarding the new users to the Hadoop data lake environment

·         Requirements gathering for creating the databases in Hive and
providing  policy based access management from the Ranger for the new Proof of
Concepts (POCs) like Veeva Insights

·         Supporting the developers for executing the adhoc jobs in Hive
environments for the existing POCs like enrollment_forecaster etc

·         HDFS home directories and Hive schema, table and column level
enforcing access bases policies management from Ranger

·         Implementation of Security and management of Active Directory based
Kerberos authentication across data lake clusters

·         Implementation of SSL for the Ambari and other HDP services in
Hortonworks environment across the data lake clusters

·         Management of Encryption and Decryption of the users data using
Ranger-KMS across the clusters of data lake environment

·         Installation and upgradation  of Jupyterhub and python packages to
support the developers for implementing the code in on-prem environments

·         working with HPC team for hardware issues  and allocation of
physical resources for the data lake environment

·         Hail- Spark implementation and analysis of UKBIOBANK datasets of
genotypes and Phenotypes

·         Installation of latest version of spark and hail and optimization
of Resources for launching datasets with huge size of data

·         Work with Hortonworks team for the planned upgradation of HDP
version from 2.6 to 3.0

·         Support and maintenance of MongoDB servers in data lake

·         Source code Repository maintenance in Bitbucket

 

In addition to the above tasks, the resource will also perform the following
AWS activities

 

·         Support of Cloudbreak server in AWS for the Hortonworks CB
Deployment

·         Support of software upgrades for Cloudbreak,HDP packages
installation in AWS Cluster

·         Support Data scientists for any technical issues during the
execution of Spark-Hail jobs in Cloudbreak AWS cluster

·         Setup of latest versions of Spark and Hail in AWS spark cluster

--

Name
Email @
Skills
Resume

Save me for future


Check Similar Jobs



Create Job Alerts



Recent Gigs

Design and deploy your website to Digital Ocean-($299)
Design and deploy your website to Digital Ocean.
About.
Not rated
Design and deploy your website to AWS.-($399)
Design and deploy your website to AWS.

About Me .
Not rated
Migrate your current website to AWS-($99)
Migrate your current website to AWS
About Me : I am a .
Not rated
1 article for your business-($2)
I will write 500 words article for your business/blog
.
Not rated
15+ private blog network links-($2)
I will promote your website with links from private network .
Not rated



Recent Projects

TV Aerials Sheffield-($1000-$50000)
Experienced TV Aerial Engineer Needs For Sheffield Area Shef
Independent Fright Dispatcher Looking For Owner operators-($0-$0)
If you need a Reliable Dispatch Service that will find YOU t
Personal Assistant-($500-$1000)
As a personal assistant (PA) you will work closely with seni
Freelance assistant-($100-$500)
hi, I need a freelance assistant for various tasks. It inclu
Freelance Website Developer -($1000-$2000)

This role is perfectly suited to a recent graduate loo