Download dictionaries #

All dictionaries can be downloaded from the SFTP server storage.apidq.io (port 65110).

Dictionaries are stored in folders named after the database schema, and the file name contains the release date: <schema>/<schema>_<YYYYMMDD>.dump, for example ru/ru_20260907.dump.

The exact versions of the dictionaries required for a particular ApiDQ version are indicated in the block Requirements for versions of dictionaries on the page changelog.

There are several ways to connect to SFTP:

CLI - this stands for Command Line Interface. All modern operating systems come equipped with an SFTP CLI program, allowing users to type out text commands to communicate with an SFTP server.

GUI - or Graphical User Interface, which allows you to list, download, and upload files using a graphical interface which is friendlier and supports the mouse or trackpad as an input device to allow drag & drop to upload and download files. The most popular SFTP GUI clients are Filezilla and Cyberduck.

Programming libraries - There are a number of standard programming libraries, for pretty much any programming language, that let programmers interact with SFTP servers in their code.

CLI #

Create a folder for dictionaries and go to it - the restore commands in the following instructions are run from this folder.

mkdir -p ~/apidq/dumps
cd ~/apidq/dumps

Connect to the SFTP server

sftp -P65110 sftp_login@storage.apidq.io

Replace sftp_login with the login to access the SFTP server

After connecting, you will see the dictionaries available to you and their versions.

sftp> ls

by        kz        ru        services  ua

sftp> ls services

services/services_20241030.dump

Download the dictionaries you need

sftp> get services/services_20241030.dump

Fetching /services/services_20241030.dump to services_20241030.dump
/services/services_20241030.dump                             100%   25KB 338.5KB/s   00:00

Non-interactive download #

For downloading from scripts it is convenient to use lftp: it takes the password from an environment variable and does not re-download a file that is already complete.

sudo apt install lftp
mkdir -p ~/apidq/dumps

export LFTP_PASSWORD='sftp_password'
lftp -u sftp_login --env-password -p 65110 sftp://storage.apidq.io \
  -e "set sftp:auto-confirm yes; get -c -O ~/apidq/dumps ru/ru_20260907.dump; bye"

Checking the downloaded file #

The dictionary of Russian addresses takes several gigabytes, so after downloading make sure the file is complete and not corrupted. The command reads the whole archive (several minutes for the ru dictionary) and does not change anything in the database:

cd ~/apidq/dumps
pg_restore -f /dev/null ru_20260907.dump && echo OK

The pg_restore -l command is not suitable for this check: it reads only the table of contents at the beginning of the file and will not notice that the download was cut off halfway.

Run this check before dropping the old dictionary from the database - see Update and rollback

GUI #

WinSCP is an excellent FTP and SFTP client for Windows. You can start copying files between a remote server and local computer using SFTP, FTP, WebDAV, SCP, or S3 protocols.

FileZilla is a minimalist FTP solution that is free to use. This open-source solution is available under the GNU GPL license and supports FTP over TLS and SFTP. It is a secure and fast cross-platform FTP client with plenty of valuable features.

Cyberduckis an excellent libre server that also works as a cloud storage browser on Windows and Mac devices. It supports FTP, SFTP, Amazon S3, WebDAV, OpenStack Swift, Azure and OneDrive, Backblaze B2, Dropbox, and Google Drive.