Open in app

Sign In

Write

Sign In

Harshit Singh
Harshit Singh

1 Follower

Home

About

Mar 4

Batch processing with PySpark and designing ETL workflow using Google Dataproc

PySpark is an API for Apache Spark to read, process, and load data, providing the interface with Resilient Distributed Datasets (RDDs) and Python to work with. RDDs are used in the prior versions of Spark which now is done using dataFrames. It is used in Batch processing the data in…

Dataengineering

3 min read

Batch processing with PySpark and designing ETL workflow using Google Dataproc
Batch processing with PySpark and designing ETL workflow using Google Dataproc
Dataengineering

3 min read


Jul 17, 2020

Kubernetes cluster on AWS using EKS services

Management is one of the most challenging task in the deployment. AWS EKS handles the management of worker nodes by themselves. Hence we can utilize our time in our business without the overhead of managing nodes. What is AWS EKS service ? Amazon Elastic Kubernetes Service (Amazon EKS) is a…

6 min read

Kubernetes cluster on AWS using EKS services
Kubernetes cluster on AWS using EKS services

6 min read


Jun 16, 2020

Deploying an webserver in aws using Terraform

Overview: Launch an EC2 instance having the required programs(git, Apache webserver, php) installed. Enabling these services to be started on reboot. Creating an EBS volume and mount it to the EC2 instance. Creating a separate Security Group for this instance having the SSH and HTTP port exposed. Clone the github repo…

AWS

4 min read

Deploying an webserver in aws using Terraform
Deploying an webserver in aws using Terraform
AWS

4 min read


May 26, 2020

MLOps: Deploying CNN model with Docker container

This is a deployment of an MLOPs environment in which any Convolution Neural Network(CNN) model can be checked for accuracy as per need, and will be retrained automatically by increasing no. of epochs and no. of layers. This consists of 5 independent jobs: JOB 1: Fetching the model from Github JOB 2: Launching the container with contains the required programs to train this model. JOB 3: Checking the accuracy of the model. If the accuracy is not achieved as required, this job then triggers job 4.

2 min read

MLOps: Deploying CNN model with Docker container
MLOps: Deploying CNN model with Docker container

2 min read


May 25, 2020

Transfer Learning on VGG16 Model

Normally traning a model takes huge chunk of resource and time. More so if the dataset is big as in VGG16. But with using transfer learning method, we can simply attach our dataset onto already trained model, which saves both resource and time. shHarsh/TransferLearningVGG Contribute to shHarsh/TransferLearningVGG development by creating an account on GitHub.github.com As you can see, there are 16 layers deep.

2 min read

Transfer Learning on VGG16 Model
Transfer Learning on VGG16 Model

2 min read


May 13, 2020

Deploying Jenkins pipeline using Docker container

Aim: To make a fully automated deployment of project from cloning the github repo to deployment alongwith basic testing. For building this I’ve used Dockerfile method to create custom container fully capable host jenkins alongwith programs like wget, net-tools, git auto installed. I have used Centos 7 as base os…

Devops Practice

3 min read

Deploying Jenkins pileline using Docker container
Deploying Jenkins pileline using Docker container
Devops Practice

3 min read

Harshit Singh

Harshit Singh

1 Follower
Following
  • Giorgos Myrianthous

    Giorgos Myrianthous

  • Anna Geller

    Anna Geller

  • Ben Rogojan

    Ben Rogojan

  • Zach Wilson

    Zach Wilson

  • Maxime Beauchemin

    Maxime Beauchemin

See all (10)

Help

Status

Writers

Blog

Careers

Privacy

Terms

About

Text to speech

Teams