topic Re: Unable to install poppler-utils in Data Engineering

Unable to install poppler-utils

Deloitte_DS — Wed, 16 Aug 2023 19:30:00 GMT

Hi,

I'm trying to install system level package "Poppler-utils" for the cluster. I added the following line to the init.sh script.

sudo apt-get -f -y install poppler-utils

I got the following error: PDFInfoNotInstalledError: Unable to get page count. Is poppler installed and in PATH?

If I install the same line at the notebook level, I don't get this error.

Can anyone help me with this issue and how to install system level packages at the cluster level in init scripts?

Re: Unable to install poppler-utils

Deloitte_DS — Fri, 18 Aug 2023 16:36:16 GMT

Hi Kaniz, I tried to include it in the init script but still it is showing the same error. The path I gave is "usr/bin". May I know how I can navigate to this path to check if my package is installed or not? Also want to know how i can navigate to databricks/bin/python? Also how to check the environment variables?

Re: Unable to install poppler-utils

dheeraj-cir — Mon, 09 Sep 2024 12:07:14 GMT

use a personal cluster and use

!sudo apt-get update

and

!sudo apt-get install -y poppler-utils

Re: Unable to install poppler-utils

Arunraja — Wed, 04 Dec 2024 22:26:54 GMT

I am using my personal cluster but still getting the same error

Re: Unable to install poppler-utils

kbmv — Wed, 15 Jan 2025 11:49:25 GMT

Hi below worked for me, I created an init script for my compute with below code

sudo rm -r /var/lib/apt/lists/*
sudo apt clean && sudo apt update --fix-missing -y
sudo apt-get install poppler-utils tesseract-ocr -y

Re: Unable to install poppler-utils

Raghavan93513 — Wed, 22 Jan 2025 04:08:56 GMT

Hi Team,

If you use a single user cluster and use the below init script, it will work:

sudo rm -r /var/lib/apt/lists/*
sudo apt clean && sudo apt update --fix-missing -y
sudo apt-get install poppler-utils tesseract-ocr -y

But if you are using a shared cluster. This solution would not work.

RCA:

Libraries installed via init scripts are not available for user-defined functions (UDFs) in Shared mode clusters, as the UDFs execute in a sandboxed execution environment. Python UDFs will not support dependencies installed via init scripts, as those are not available in the sandbox environment.

We need init scripts for Poppler, especially in your case, as your code (for example, pdf2image) relies on Poppler’s command-line utilities.

Solution:

For single mode cluster: Use the current functionality.
For shared mode cluster: Users can consider using alternative Python libraries that provide similar functionality to poppler-utils. Two such libraries are pdfplumber and PyMuPDF.

PyMuPDF: Screenshot 1
Pdfplumber: Screenshot 2

Hope this helps you!