Must Know Big Data Interview questions and answers

 

I am providing you some of very important questions that you will be facing as Data Engineer ,Bigdata Developer, Hadoop Developer etc.

1. What is your role in your Bigdata/Spark  project?

Here, You need to answer what is the exact role in your project .What task you basically performed and worked as a developer or data engineer or data scientist role..You could mention also your development works if you did any.

Your daily activities regarding maintenance of server,jobs scheduling ,jobs development etc are to be discussed. 

2. Which setup you are using ? is it on-premise setup or it's in cloud?

You must be having idea in which platform you are working .Is it on premise or working on cloud setup like Amazon Web service ,Microsoft Azure, Google Cloud etc.

3. Configuration of each node in cluster in which you are working with?

What is configuration set up present in your cluster in which you are working on .Your CPU core details and memory details of node etc.

4. How much data you have worked till now?

What is the data size or file size you are dealing daily .

5. Any challenges that you have faced in your big data project and how did you overcome that?

What are the challenges you have faced while working on project . You can mention any two or more if any error faced and how you optimize or eradicate that  error occurred. 

6. Did you ever face any performance challenges with your spark job? how did you optimize that?

While deploying spark jobs there are performance challenges you might face.You have mention those challenges and optimization done on that.

7. Big data distribution in which you are working currently?

Big data distribution in which you are using in project.

8. What is the cluster size in which you are working?

   Cluster size of your set up . Data nodes and Name nodes size and details etc.

9. Use case like total file size given and you might be asked about nodes configuration needed for you cluster ?

10. If worked in Spark project the what is the deployment configuration used ?

    You have to prepare for Spark submit configuration used in your deployment like

    Executors and its memory , cluster like mesos ,yarn etc used .

    
    
    
    
    
    I hope this would benefit you a lot .Now you won't get surprised to face these questions.Please prepare well these general question which you are bound to answer if you are appearing for Bigdata Developer /Hadoop Developer/Spark Developer/Data Engineer / Data Scientist Interview.Please share this as much as possible so that other can also prepare well for it.

For more updates please follow Itechshree blogs.

Happy learning!!