cancel
Showing results for 
Search instead for 
Did you mean: 
Machine Learning
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

MattM
by New Contributor III
  • 974 Views
  • 2 replies
  • 3 kudos

Resolved! Mapping Control data - Maintained by Business User

We are ingesting our data from ADLS into databricks as delta table. After raw layer we need to refer to a control\mapping layer which defines certain logic\measure definition. This would be incorporated in the subsequent silver or gold layer. This co...

  • 974 Views
  • 2 replies
  • 3 kudos
Latest Reply
MattM
New Contributor III
  • 3 kudos

Thanks for your response. Can business user without the help of any script modify any rows in the table after loading it onetime from CSV fiels?

  • 3 kudos
1 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 2584 Views
  • 2 replies
  • 32 kudos

Databricks Roadmap Azure There are a lot of excitement new features coming in 2022. I tried to put them all on one list: Unity catalog (seems that it ...

Databricks Roadmap AzureThere are a lot of excitement new features coming in 2022. I tried to put them all on one list:Unity catalog (seems that it will exists next to hive metastore and it will be possible to migrate)Control metastore, unity creatio...

  • 2584 Views
  • 2 replies
  • 32 kudos
Latest Reply
Kaniz
Community Manager
  • 32 kudos

Hi @Hubert Dudek​ , We know the world is full of choices. Thank you for choosing us! We would like to thank excellent customers like you for your support. We couldn’t do it without you! 

  • 32 kudos
1 More Replies
anu_sh
by New Contributor II
  • 1298 Views
  • 3 replies
  • 6 kudos
  • 1298 Views
  • 3 replies
  • 6 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 6 kudos

Here are the supported data types for the Feature Store:https://docs.databricks.com/applications/machine-learning/feature-store/feature-tables.html#supported-data-typesAs you can see, image is not between them, but you could use BinaryType.

  • 6 kudos
2 More Replies
MadelynM
by New Contributor III
  • 1220 Views
  • 1 replies
  • 7 kudos

2021-07-Webinar--Hassle-Free-Data-Ingestion-Social-1200x628

Thanks to everyone who joined the Hassle-Free Data Ingestion webinar. You can access the on-demand recording here. We're sharing a subset of the phenomenal questions asked and answered throughout the session. You'll find Ingestion Q&A listed first, f...

  • 1220 Views
  • 1 replies
  • 7 kudos
Latest Reply
Emily_S
New Contributor III
  • 7 kudos

Check out Part 2 of this Data Ingestion webinar to find out how to easily ingest semi-structured data at scale into your Delta Lake, including how to use Databricks Auto Loader to ingest JSON data into Delta Lake.

  • 7 kudos
User16752240150
by New Contributor II
  • 572 Views
  • 1 replies
  • 1 kudos

What algorithms does Databricks AutoML use?

AutoML presumably tries a few different algorithms while hyperparameter searching. What model types are considered?

  • 572 Views
  • 1 replies
  • 1 kudos
Latest Reply
sean_owen
Honored Contributor II
  • 1 kudos

At the moment, it's really just xgboost, and sklearn implemenations like random forests, logistic regression, and linear regression as applicable. More possibilities are coming.

  • 1 kudos
User16752239203
by New Contributor
  • 602 Views
  • 1 replies
  • 0 kudos

How can I use Non- Spark related libraries like spacy with Databricks and Spark

I have an NLP application that I build on my local machine using spacy and pandas, but now I would like to scale my application to a large production dataset and utilize the benefits of sparks distributed compute. How do I import and utilize a librar...

  • 602 Views
  • 1 replies
  • 0 kudos
Latest Reply
sean_owen
Honored Contributor II
  • 0 kudos

It depends on what you mean, but if you're just trying to (say) tokenize and process data with spacy in parallel, then that's trivial. Write a 'pandas UDF' function that expresses how you want to transform data using spacy, in terms of a pandas DataF...

  • 0 kudos
User16826994223
by Honored Contributor III
  • 1275 Views
  • 1 replies
  • 0 kudos

Muliple Where condition vs AND && in Pyspark

.where((col('state')==state) & (col('month')>startmonth)I can do the where conditions both ways. I think the one below add readability. Is there any other difference and which is the best?.where(col('state')==state).where(col('month')>startmonth)

  • 1275 Views
  • 1 replies
  • 0 kudos
Latest Reply
User16826994223
Honored Contributor III
  • 0 kudos

You can use explain to see what type of physical and logical plans are getting created . This is the best way to see difference , but as mentioned in the question , it should give the same physical plan

  • 0 kudos
User16789201666
by Contributor II
  • 1846 Views
  • 2 replies
  • 0 kudos

What is Databricks' model deployment framework?

How do you do deploy a model in Databricks.

  • 1846 Views
  • 2 replies
  • 0 kudos
Latest Reply
RonanStokes_DB
New Contributor III
  • 0 kudos

The following resources provide more detail on this:Databricks model registry example notebook: https://docs.databricks.com/_static/notebooks/mlflow/mlflow-model-registry-example.htmlDatabricks model lifecycle - https://docs.databricks.com/applicatio...

  • 0 kudos
1 More Replies
Labels