This software has been known to run very well on Ubuntu 18.04 and 20.04 servers. We have used Contabo VPS with 8gb RAM and 200gb SSD, which is crazy cheap at 6 euros a month. We also use AWS S3 and CloudFront, which will cost almost nothing until your map is a big success (alternatives would be possible but you’d have to hack it a little). We cannot ensure that any installation will work, but the steps below will probably get you there.
The software is mounted in docker containers, which sounds complicated, but in fact it’s very easy to set up. The guide has been written so that amateurs can use it. So there will be some redundancy for more experienced sys admins, sorry about that.
ASSUMPTIONS
This guide assumes you are logged into the server as root. Otherwise, add “sudo” to every command. Commands are shown in italics.
It also assumes you have a dedicated URL for the project. If you don’t want to buy a new one, make a subdomain on top of the old one, by adding an “A Record” using the domain manager of your existing URL (the result will be something like this: subdomain.YOUR_URL.org). If you want to add the control panels, Traefik and Portainer, you also have to create subdomains for them. But they are not necessary, you can skip it.
Whenever you are asked to run a program with the flag -v or –version, that’s just to check that the program is installed correctly and which version
PREREQUISITES
1. Take a deep breath. There are a lot of things to do. You’ll be here for a couple of hours.
2. Install git, docker and a few other things on the server, using these commands:
apt update
apt upgrade
apt install net-tools [we’ll be using this to run net-stat]
apt install build-essential
apt install git-all
apt remove docker docker-engine docker.io [if these might be present]
apt install docker.io
systemctl start docker
–> If docker install fails with this message: “The following packages have unmet dependencies: docker.io : Depends: containerd (>= 1.2.6-0ubuntu1~)”, then run these commands:
apt remove docker* containerd*
apt install containerd
apt install docker.io
systemctl start docker
–> If the docker.io installation fails to start with this message: “Failed to start docker.service: Unit docker.service is masked”, then run these commands:
systemctl unmask docker
systemctl start docker
3. Now install docker-compose with these 2 commands (you might check the Docker version and get a more recent one):
curl -L "https://github.com/docker/compose/releases/download/1.29.1/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
chmod +x /usr/local/bin/docker-compose
CHECK IT:
docker-compose --version
If you ever need to remove docker-compose, run this:
rm $(which docker-compose)
4. Install python3 (which is probably already there):
CHECK IT FIRST:
python3 --version
If you don’t have it, instructions here: https://phoenixnap.com/kb/how-to-install-python-3-ubuntu
5. Set up your S3 bucket on Amazon Web Services:
— Name the bucket with your PROJECT name. Uncheck the “Block all public access” box when you create it. You are going to need your S3 ACCESS KEY, you S3 SECRET KEY and your chosen region later on, so save those as you get them (you can create the Access and Secret key by clicking your account name at upper right, choosing Security Credentials in the dropdown, and follow the instructions).
— Then make three folders: assets – geodatadir – pgdumps
— In “assets” make five folders: audios – icons – images – settings – tiles
— Now choose “Services” from the AWS navbar (upper left) and pick CloudFront (you can search for it if it’s not immediately visible). It’s a Content Delivery Network with servers all around the world, you have to subscribe to it. It will accelerate the delivery of images of all kinds, including tiles if you use them.
— Once you are on the starting page of your CloudFront account, choose “Create distribution” and then select the name of your bucket. For “Origin path” put “/assets”. Further down , choose “Redirect http to https”. It will ask you if you want WAF security protection, we usually choose no.
— Scroll down and click “Create distribution”. Now you will see a table with a column that says “Domain name”. Copy that, you’ll need it soon.
— Finally, go back to the root of your bucket, which has your PROJECT name. Click “Permissions” and scroll down until you get to “Cross Origin Resource Sharing (CORS)”. Click “edit” and past in this CORS policy:
[
{
"AllowedHeaders": [
"*"
],
"AllowedMethods": [
"GET",
"HEAD"
],
"AllowedOrigins": [
"*"
],
"ExposeHeaders": [],
"MaxAgeSeconds": 3000
}
]Save it and you are done with S3 for the moment.
6. Finally, you will need a database client on your local machine. We use DBeaver: https://dbeaver.io. You can install it and have it ready.
INSTALLATION
1. Go to https://github.com/crystalball-mapkit/crystalball and copy the address where it says “Code”. Then cd to your home folder on the server (“cd /home”) and download the whole repository with the “git clone” command. For convenience, just do this:
git clone https://github.com/crystalball-mapkit/crystalball.git
When this is done you will have a folder named “crystallball” in your /home directory.
2. Pick a PROJECT name, which should be short because you will use it many times. Change the name of the folder on the server by running this command, replacing “PROJECT” with your project name:
mv crystalball PROJECT
3. Also clone traefik which will be the proxy listening on port 80:
git clone https://github.com/crystalball-mapkit/traefik-map.git
There’s nothing else to do with traefik at this point.
4. Now run netstat:
netstat -tulpn
You’ll probably see a line like this in the output:
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 4722/apache2
We are going to install traefik later on, and it won’t work if apache is listening on port 80. So let’s get rid of it:
apt purge apache2
apt autoremove
5. Now cd to /home/PROJECT and run
ls -a
You will see that there is a hidden file called .env-template. You can open it with this command:
nano .env_template
Now you are in the nano editor. Notice that you have to move around in here with the arrow buttons, and you have to delete by using the BACKSPACE or DELETE key.
This .env file will set up your entire deployment. Fill it in very carefully. PROJECTNAME is the name of your project folder. PROJECT_VERSION should be left unchanged and DOCKERID should be left as crystalballmapkit. DOMAIN_NAME is the URL you will be using for your project (without https://). Skipping down, set your GEOSERVER_ADMIN_USER and GEOSERVER_ADMIN_PASSWORD and write those down somewhere. Same for the POSTGRES_USER and the POSTGRES_PASSWORD, you will be using those to connect to the database through DBeaver. You can leave the outer port set at 55432 (but for a second map on the same server you will have to change that to 45432). All the S3 info needs to be filled in. Finally, choose a password for API_JWT_SECRET, and make a decision whether you want the site to have open registration (means, anyone can register just by entering their name and email). It depends on your needs, but usually you are going to want to put
USERS_OPEN_REGISTRATION=False (you will be able to change this setting later if you want)
Now hit ctrl+O. You will see that it says “File Name to Write: .env_template. Erase the .template part and then hit ENTER. It will ask you, Save file under a different name? Just type Y and that will do it (it looks like you should type Yes but it will save as soon as ‘Y’ is entered, and your “es” will mess up the file). To get out of the nano editor when you’re done, hit ctrl+X.
Now, what happens if you make a mistake and save your file as .env_template?
What will happen is that the completed .env_template will become public. The .env file itself is listed in a thing called gitignore and therefore will not be transferred by git to the public repository. However the template is included in the public repo for all eyes to see. So, as soon as you update your repo, some malicious bot will almost instantly steal your s3 password and start using a LOT of storage space. So be careful with this step!
[ skip 6 and 7 if installing second map ]
6. Cd to /home/traefik-map and run ls -a again. You will see another .env file – not a template this time, just .env. Fill in your email (that’s important) and (optionally) add the domains for traefik and portainer. The address and URL goes directly after the ‘=’ sign – you don’t need to have those square brackets around them, it’s like this:
EMAIL=myaddress@host.org
After filling that in, just save using ctrl-o followed by ENTER, then ctrl-x to exit.
7. Now, while still in the traefik folder, start the containers with this command:
docker-compose up -d
That’s going to pull the images and build the containers for the traefik load balancer.
8. Next, cd back to your project folder. Here you will run 2 commands. First:
docker-compose build client geoserver api rclone db pgdumper
That will pull the docker images before your eyes. So it will take some time. Some red ink appears, but it has never yet had consequences. After it’s done, run this:
docker-compose up -d
And now you should be able to see your site online! If the https does not work immediately, don’t worry, sometimes it takes an hour or two to kick in.
9. Now you should be able to connect to the PROJECT database using the open-source DBeaver client. So fire up DBeaver. Choose “New” from the File menu, select “Database wizard” and then PostgreSQL. “Host” is the server’s ip number, “Database” is PROJECT, “Port” is 55432 (or whatever you entered in the .env) and the username and password are the ones you defined in app/config/docker-env/db.env
Your new db will appear at the left side of DBeaver. Open it, because otherwise you can’t tell if it works. If it does, open Schemas, Public, Tables – and then you will see the data tables. Right click on one and the context menu will allow you, among many other things, to “View data.” You can try that on some of the tables to see what’s in there.
10. You should now be able to connect with geoserver (YOUR_URL/geoserver will do it). Your login will be whatever you wrote in the .env file. Before you hit ENTER, check “remember me” for convenience – it will save you re-entering the password every time.
Click Data / Workspaces / Add new workspace. “Name” should be “workspace1” and “URI” should be “http://workspace1”. Then click Save.
Now click Data / Stores, and then click on “Store Name”. Here you are going to choose POSTGIS database from the “Vector Data Source” list, and then put in the name of your db, that is, PROJECT, right at the top where it says Data Source Name. Description is “POSTGIS connection.” It’s not intuitive, but the host will be PROJECT_db (so you just add _db to whatever the project name is). “Database is, again, just PROJECT. Port: 5432 (note for second map: even if you are installing a second map and you wrote 45432 or maybe 65432 in the .env, you should still use this four-figure value, 5432). Schema: public. User and psswrd are the ones you defined for your database. Then click Save.
Finally, return to the main menu on the left click “Layers” and then select “Add a new layer.” Choose “workspace1:PROJECT” (there should only be one choice; even if it says something different, choose it). You will then see a list of layer names, including “SequelizeMeta” and a number of file beginning with underscores. You never want to do anything with those entries, or the corresponding tables. Instead, go to the last line, “html_posts”, and click on “Publish.” Scroll down to the “Bounding Box” section, and click “Compute from data” for the first choice, and “Complete from native bounds” for the second one. Then hit Save at the bottom of the page.
What you just did was to install the data table that will store user posts. All other data tables will be installed in the same way, except that in all other cases, you have to upload an initial file (we’ll cover that later).
11. Now we are going to set up something very useful for everyday mapmaking, which is an editable configuration file outside, of the git repository, so you don’t have to go through the whole git routine when you want to change a tiny detail. This will also make it much easier to pull updates from the core repository. Start by downloading this file:
https://s3.us-east-2.amazonaws.com/ecotopia.today/Data/app-conf.json
You will then take that file, store it in a separate folder on your local machine, and upload it to your S3 bucket, in the folder PROJECT/assets/settings. Once you’ve uploaded it, click on the file and take a note of the S3 URI, which will be like this: s3://PROJECT/assets/prod/settings/app-conf.json. Also note the whole URL, which should be something like this: https://PROJECT.s3.YOUR_REGION.amazonaws.com/assets/prod/settings/app-conf.json
12. Now you are going to go back on the server. You want to open this file: PROJECT/app/client/src/main.js. On line 51 (or thereabouts, it could change in the future). you will see a line like this:
fetch(‘https://s3.us-east-2.amazonaws.com/ecotopia.today/assets/prod/crystalball/settings/app-conf.json’)
You put your app-conf URL from S3 in there. This means that the program will now fetch its configuration file from S3. But of course, the active program is not in this file, it’s in a docker container. So once you have made the change, you have to rebuild the docker container. For that you cd to the PROJECT folder on the server, and you run this command:
docker-compose up -d --force-recreate --build client
That may take a while. But when it is done… you have finished the installation!