Run Pinterest’s Querybook Locally
Pinterest Querybook is an open source project, and
Querybook is a Big Data Querying UI, combining collocated table metadata and a simple notebook interface.
I was experimenting its lineage feature, and found out the project doesn’t work out of the box as advertised.
You probably ask, “why not send a PR to ask a review, and have your solution merged? “ . Since there is no mention about those issue I faced, I thought maybe it’s just me? I thought it could be easier for other people in case they run into the same issue and ask Medium for a help.
I cloned the project, then run make in the root directory. After the build process finished, the docker images started to run, I saw some errors
Python Modules such as pyhive, Boto3, etc Not found
About this error, related to the Python modules, such as Pyhive, Boto3, not found, the documentation said those dependency are needed to be added based on how you will use it. To me, the simplest way to get it going is to add it into this line in the Makefile
dev_image:docker build --pull -t querybook-dev . --build-arg PRODUCTION=false --build-arg EXTRA_PIP_INSTALLS=dev.txt --build-arg EXTRA_PIP_INSTALLS=extra.txt
The extra.txt includes some extra Python dependency package.
`gevent` Not Found
Another issue I came across is that gevent not found. This package is defined in the base.txt, I am not sure why this error, but I have to add it into the dev.txt to get this issue solved.
Or this gevent is actually used by the watchdog package, and there is an typo for this watchdog_gevent in this line. It should be watchdog-gevent
watchmodo Not Found
Once you solve the above issues, there are two containers quickly exited from running, one the scheduler, the other is the worker. You read the logs, and have the watchmedo not found error.
the watchmedo is used in one of these lines.
It has been defined in the dev.txt, but apparently has not been respected. Let’s add them into the base.txt
watchdogwatchdog-geventwatchdog[watchmedo]
Now you should be able to get your Querybook running locally.
