Featured Member Interview - February 2023
Name: Aviral Bhardwaj ⭐️
Community nickname: @Aviral Bhardwaj
Pronouns: He/Him
Company: Knowledge Lens Pvt Ltd, Bangalore, India
Job Title: Senior DevOps Engineer
Databricks Certifications
Academy Accreditation - Databricks Lakehouse Fundamentals
Academy Accreditation - Platform Administrator
Databricks Certified Data Engineer Associate
Databricks Certified Associate Developer for Apache Spark 3.0
Tell us about your professional journey to date.
I first began my entrepreneurial journey in 2016, attempting to launch my own business as a web developer and content writer. However, it quickly became apparent that greater expertise was needed to be successful in this endeavor. With this realization in mind, I applied for the postgraduate Big Data Analytical Diploma Course at CDAC Pune and was accepted. During the course of my studies at CDAC, I obtained a software engineering position at Knowledge Lens Pvt Ltd Bangalore where I have been employed for the past 3.7 years. My current responsibilities include working with AWS, Linux, Databricks Data Science and Engineering, SQL Analytics, Databricks Admin and Unity Catalog extensively.
What is your favorite thing about your current job/role?
My favorite part of the job is working in Databricks and solving data-related problems for our clients and providing them with the necessary solution so that they can move and compute petabytes of data. For example: how to connect with different clients with Databricks - sometimes data is received in batch form and at times in small chunks; how we can set up DLT for the user; implementing new Databricks runtime version and checking how it is working better than earlier version.
What are three words your co-workers would use to describe you?
Super experienced, reliable, and helpful.
Do you have any mentors in your professional life? What is the best advice you’ve gotten from your mentors?
Throughout my professional career, I have been fortunate enough to have mentors. My team leads are Vignesh Prabhu and Kishore. I have my manager Harman Singh, Sampath Shetty, & Sudheesh Narayanan and my friends Nishant, Kavya, Sidharth, & Krishna - all are super helpful and it is a pleasure to work under their guidance.
Promising less and delivering more are the best pieces of advice I have received from my mentors.
When and why did you start using Databricks?
I started working in Databricks in 2019 and was amazed at how clusters are created, and how fast data is in computing, and the speed at which these things move. As our company previously used Hadoop, processing data was very slow, so we decided to move to Databricks for better security and governance.
Do you have a favorite Databricks feature? Something that makes your life a lot easier? Or more interesting?
My favorite feature is Delta; it saves us from calamity if data is deleted we have Time Travel and versioning features there. It also comes with a Schema enforcement feature that is helpful and the ACID transaction feature is more than what we need in Databricks.
Is there a Databricks feature you want to see in the future?
If Databricks comes up with AZ as Auto in the advanced section of cluster configuration then it will be awesome because most of the time spot instances are not available in certain AZ and it causes job failure or if the cluster is starting it will keep starting until the required spot instances availability. If Databricks implements this feature, it will be very helpful for us to continue running our Databricks jobs.
When and why did you start using Databricks Community?
After completing my certificates, I started posting about Databricks on LinkedIn and our company's Medium Blog but soon I realized that I can help more people. That’s when I found the Databricks Community where I can help or contribute to more than 80K+ users.
What’s your favorite thing about being in the Databricks Community?
In the Community, there are experienced peers who have explored every tech stack in Databricks, and I love to read their posts. Also, the newly released articles that are posted by the content management team is appreciable.
Pivoting a little: What’s your favorite hobby?
I love to write technical blog posts and create educational courses. I have created one on Python in Udemy and am working on releasing a Spark course shortly.
Where do you see yourself in 3 years?
I can see myself as a good data engineer with the required skill set and contributing a lot in a Databricks community.
Social Media Handles:
https://www.linkedin.com/in/aviralb/
https://medium.com/@aviral_39691
https://www.facebook.com/aviral.bhardwaj.585
Thank you for sharing a bit about your life and professional journey, and for letting us 'take a piece of greatness' from you @Aviral Bhardwaj 😊
Let Aviral know what questions you have for him below!
