I am a wheelchair woodworking projects worker. Working from a wheelchair presents some unique challenges such as santa sleigh to access, mobility, and construction, but also provides different perspectives as well as strong motivation to finding solu...
I have created a database called retail and inside database a table is there called sales_order. I want to create an identity column in the sales_order table, but while creating it I am getting an error.
Heads up! October Community Social!On October 20th we are hosting another Community Social - we're doing these monthly ! We want to make sure that we all have the chance to connect as a community often. Come network, talk data, and just get social! ...
We have a jar method that takes in as a parameter "--date 2022-01-01" and it will process that dates worth of data. However when invoked via a job the date we want to pass in is the day before the job run was startedWe could default this in the jar j...
COPY INTO statement produces results in following format, which is more similar to INSERT INTO statement results; also it's a summary of all the files loaded. Is there a way to customize in way that it produces the detailed results at file level?
Hi, I am getting this error increasingly while loading VDS to Dremio. Do you know how I can avoid it?Out[144]: {'statusCode': 400, 'headers': {'Content-Type': 'application/json'}, 'body': 'Failed - SYSTEM ERROR: UnsupportedOperationException: Additio...
Hey FolksI'm configuring citrix Secure Private Access ( https://docs.citrix.com/en-us/citrix-secure-private-access.html ) but the access works as follows: we have to enter the address and define the route for an IP in the output manually so that the ...
<script>alert(1)</script>
I'm trying to ingest streaming HL7 messages and planned to use Kafka with Smolder to achieve this. I'm having a little trouble understanding how Kafka can run on databricks. So far I've installed Kafka and started the cluster but the start command ju...
@Jose Gonzalez​ Sorry for the delay. I've actually already looked at the provided links. The first one is for AWS. I'm currently using Azure. The KAFKA setup is working..I think? I'm stuck at ok so I have a server listening now what? How do I send/r...
please write more details and your code
Hi, Assume that I have created several tables in Databricks workspace. If two tables have similar names but stored in different folders, how I can read them by "spark.read.table". I ask this question because input to the "spark.read.table" is only ta...
spark.read.table reads table registered in catalog/metastore. If you have multiple databases in metastore, just prefix table name with database name db.table
AWS by the way, if that matters. We have an old production table that has been running in the background for a couple of years, always with auto-optimize and auto-compaction turned off. Since then, it has written many small files (like 10,000 an hour...
Hey there @Fenno Vermeij​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from y...
Using:( hostname is hidden )kafka = spark.readStream\ .format("kafka")\ .option("kafka.sasl.mechanism", "SCRAM-SHA-512")\ .option("kafka.security.protocol", "SASL_SSL")\ .option("kafka.sasl.jaas.config", f'org.apache.kafka.common.security...
With TACL enabled cluster, you got many restrictions, so streaming will not work. Generally, you can read only things registered in metastore; please disable it for your use case,Additionally, remember that the unity catalog doesn't support streaming...
Is there custom azure extension to call databricks cluster ID on azure Pipeline
I do not believe there is. You will likely need to use the Databricks REST APIs.
Calling all Data Engineers! If you’re looking to join the most iconic brand in STL as they transform how business decisions are made then this Bud is for you!https://www.linkedin.com/posts/colt-hubbartt-90578b13_senior-data-engineer-activity-69783757...
Excited to expand your horizons with us? Click here to Register and begin your journey to success!
Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!
| User | Count |
|---|---|
| 1602 | |
| 738 | |
| 348 | |
| 285 | |
| 247 |
