Forum Replies Created
-
Posts
-
The concept of clusters vs. notebooks was confusing. I didn’t realize a notebook doesn’t run unless the cluster is active.
Here are a another questions that came up during my interviews:
“Tell us about a time you had to debug a failing Spark job in production. How did you approach it?”
In my answer, I followed the STAR format:
Situation: Daily ETL job was failing intermittently
Task: Identify root cause and fix without data loss
Action: Checked job logs and Spark UI, traced a data skew issue due to a join on a non-partitioned column
Here’s a mock interview question I was recently asked:
“How does Delta Lake handle ACID transactions, and what are some scenarios where you would recommend using Delta over traditional Parquet tables?”
My answer was focused on the transaction log (_delta_log), schema enforcement, and time travel features.
I also mentioned its advantages in streaming + batch workflows.I’ve also seen cost reductions by monitoring storage growth regularly and archiving old data that’s no longer queried frequently.
We now run a monthly review of unused tables and large result caches.
Also, switching to transient tables where possible helped lower long-term storage costs.Let me know if anyone wants to collaborate on setting up resource monitors and alerts — we did it via Snowflake’s UI and some simple SQL scripts.