Options
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-13-2023 01:13 AM
Thank you, this was exactly what I've been looking for! A way to install packages dynamically. I can confirm it works great for other methods like pypi. If any moderators see this please add this method to the documentation as it is perfect for CICD. My example below.
locals {
requirements = split("\n", trimspace(file("requirements_frozen.txt")))
}
task {
task_key = var.task_name
job_cluster_key = "temp"
spark_python_task {
python_file = var.python_file
source = var.task_source
parameters = var.parameters
}
dynamic "library" {
for_each = toset(local.requirements)
content {
pypi {
package = library.value
}
}
}
}
