- 10135 Views
- 11 replies
- 10 kudos
Hi Everyone,I am getting the following error when running a SQL query and do not understand what it means or what can be done to resolve it. Any recommendations?View DDL:CREATE VIEW myschema.table (
accountId,
agreementType,
capture_file_name,
...
- 10135 Views
- 11 replies
- 10 kudos
Latest Reply
Hi @Michael Okulik Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
10 More Replies
- 19607 Views
- 3 replies
- 2 kudos
Hello everyone,I am trying to determine the appropriate cluster specifications/sizing for my workload:Run a PySpark task to transform a batch of input avro files to parquet files and create or re-create persistent views on these parquet files. This t...
- 19607 Views
- 3 replies
- 2 kudos
Latest Reply
If the data is 100MB, then I'd try a single node cluster, which will be the smallest and least expensive. You'll have more than enough memory to store it all. You can automate this and use a jobs cluster.
2 More Replies
- 5617 Views
- 3 replies
- 4 kudos
In Spark, is it possible to create a persistent view on a partitioned parquet file in Azure BLOB? The view must be available when the cluster restarted, without having to re-create that view, hence it cannot be a temp view.I can create a temp view, b...
- 5617 Views
- 3 replies
- 4 kudos
Latest Reply
Here is what worked for me. Hope this helps someone else: https://stackoverflow.com/questions/72913913/spark-persistent-view-on-a-partition-parquet-file/72914245#72914245CREATE VIEW test as select * from parquet.`/mnt/folder-with-parquet-file(s)/`@Hu...
2 More Replies
