โ07-09-2023 10:52 AM
Hello!
I am contacting you because of the following problem I am having:
In an ADLS folder I have two items, a folder and an automatically generated Block blob file with the same name as the folder.
I want to use the dbutils.fs.mv command to move the folder to another folder but as the Block blob file has the same name it is also being moved together with the folder and this is causing me a problem.
Would it be possible to somehow just specify that I want to move the folder?
Thank you very much in advance!
โ07-10-2023 02:12 AM
Thanks @SaraCorralLou for sharing snapshot, for this you have to handle the code itself while moving:
def move_folders(filepath, destination_path):
"""
This function moves all folders from the specified filepath to the specified destination_path.
Args:
filepath (str): The path to the source folders.
destination_path (str): The path to the destination folders.
"""
# Iterate over all files in the source directory.
for file in dbutils.fs.ls(filepath):
# Check if the file is a directory.
if file.isDir():
# Move the directory to the destination directory.
dbutils.fs.mv(file.path, destination_path,True)
# Print a message to indicate that all folders have been moved.
print("All folders have been moved successfully.")
โ07-09-2023 07:23 PM
Hi, @SaraCorralLou can you share a snapshot for more clarity?
โ07-10-2023 01:13 AM
Yes, sure.
This is the situation. The second file is this Block Blob file that does not have extension. The folder is the one that we want to move but when we do dbutils.fs.mv is moving both.
โ07-10-2023 02:12 AM
Thanks @SaraCorralLou for sharing snapshot, for this you have to handle the code itself while moving:
def move_folders(filepath, destination_path):
"""
This function moves all folders from the specified filepath to the specified destination_path.
Args:
filepath (str): The path to the source folders.
destination_path (str): The path to the destination folders.
"""
# Iterate over all files in the source directory.
for file in dbutils.fs.ls(filepath):
# Check if the file is a directory.
if file.isDir():
# Move the directory to the destination directory.
dbutils.fs.mv(file.path, destination_path,True)
# Print a message to indicate that all folders have been moved.
print("All folders have been moved successfully.")
โ07-10-2023 05:08 AM
Great! Thank you so much.
โ07-10-2023 06:53 AM
โ07-12-2023 02:14 AM
Thank you for your answer.
At the end I had to add one condition more because when you have these to elements (the folder and the block blob file) Databricks "does not understand" that is a directory so I added this condition:
if "ckd_tvhdep_parquet" in file.name and file.isDir() == False to remove first the file and after that I could move the folder. Because at that time, when the file was no longer there, Databricks did identify it as a folder.
Thank you very much anyway, your answer was very useful!
โ07-11-2023 03:34 AM
Thank you for posting your question in our community! We are happy to assist you.
To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?
This will also help other community members who may have similar questions in the future. Thank you for your participation and let us know if you need any further assistance!
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโt want to miss the chance to attend and share knowledge.
If there isnโt a group near you, start one and help create a community that brings people together.
Request a New Group