Developers can save time by using ready-made solutions to build new products. Most startups concentrate on solving one problem and doing it well. We have at least three cloud providers to host our code. There are multiple choices on what to use to build a frontend for our next project. For example the VueJS and ReactJS projects with broad open source communities.
In this article, I'll show you an example of how to build an URL shortener using a modern open source technology stack.
Here's an overview of the backend architecture we are going to build including Ory Kratos, Supabase, and Ory Oathkeeper:
You can find the source code for this project on GitHub.
What we will use
- Ory Kratos to manage identities and users. We'll use an open source self-hosted version for this tutorial, but for production I recommend the Ory Network - you get a fully featured Ory Kratos instance deployed and ready to use for free.
- Supabase is an open source alternative to Firebase. Supabase Database comes with a Postgres database, a free and open-source database that is considered one of the world's most stable and advanced databases. We'll use Supabase as database for our URL shortener.
- Ory Oathkeeper would be a great example of applying Zero Trust architecture for our project. We'll use it as identity and access proxy.
- Postgres is a powerful, open source object-relational database system with over 30 years of active development that has earned it a strong reputation for reliability, robustness, and performance.
- golang-migrate to perform database migrations.
Backend choices
I'm a huge fan of the Go programming language, and I've been coding using this language since late 2014. I love the simplicity of the Go language design and the ecosystem around it.
- It's good to perform static code analysis to make your backend systems more robust and stable. golangci-lint is a feature-rich linter that gives you feedback about your code. You can find a lot of linters available.
- Strongly typed programming language with static data types. One does not need
to write tests to check that your code will never mess with data types.
int i
makesi
an integer forever. - It's easy to follow SOLID principles and clear architecture using Go.
- In addition, Go is fast and well-scaled programming language
Database migrations
Database migrations should be used once you use any relational database management system (RDMBS) in your project because you need to change the database schema from time to time. It helps you track the different versions of your schema and easily perform forwards and downwards migrations. In the pythonic world, it's easy to decide what tool to use for schema migrations because it's usually comes out of the box with the selected framework. For instance, we have flask-migrate for flask and Django migrations for Django. Since Go uses the UNIX philosophy to build the architecture of your project, you need to choose:
- a HTTP router for your endpoints.
- an ORM or a library to work with the database.
- a tool to perform migrations.
The Go programming language has at least two tools for schema migrations. I used
both goose
and migrate
, and for this project, I decided to go with
migrate
. Migrate supports more databases, and I want to make the DB layer in
this project database agnostic.
My requirements for the Go migration tool:
- Plain SQL migrations support. I don't want to learn an additional filetype formats. I know SQL and I know how to create tables. That's enough. Unlike Django, Go does not have any good Active Record pattern implementations. I hope that there will be more ORMs and better tooling once is Go 1.18 released - generics are on the way.
- Support of open source RDBMS like Postgres, MySQL (and all their forks), Oracle (I don't use it yet).
- Programmable API or a shorthanded way to apply migrations.
- Upward/Downward support.
Gin
A URL shortener is a lightweight service, and it would be ideal to have a simple enough package to build an HTTP API around it. You have several options on what to choose to solve this issue, and the most popular frameworks are:
Go-kit is an excellent framework for building a complex system with many micro-services. Even a simple net/http with HTTP router would be enough for our example. Usually, I toss a coin when I choose between Echo and Gin, so I use Gin in this project.
I'm huge fan of gRPC when I build APIs with Go and I used it a lot previously.
Also, I'm huge fan of the Echo framework. I chose Gin because of it's
simplicity, convenience, and feature rich support. A simple net/http
would be
enough to build this project with
httprouter but Gin is nice to
work with.
Okay. Let's start hacking, shall we?
Configuration package
Create a new folder for your project, and create a file main.go
:
Defining database schema
The database for our URL shortener should have the following tables:
url
table to store shortened URLs.url_view
table to store views. This information will be useful to build additional reports for users about top referrers, urlviews or something else.
CREATE TABLE IF NOT EXISTS url (
id SERIAL PRIMARY KEY,
url VARCHAR(255) NOT NULL DEFAULT '',
hash varchar(10) NOT NULL DEFAULT '',
created_at TIMESTAMP WITH TIME ZONE DEFAULT CURRENT_TIMESTAMP,
expired_at TIMESTAMP WITH TIME ZONE,
owner_id VARCHAR(36) NOT NULL DEFAULT ''
);
CREATE UNIQUE INDEX idx_url_hash ON url(hash);
CREATE TABLE IF NOT EXISTS url_view (
id SERIAL PRIMARY KEY,
referer VARCHAR(255) NOT NULL DEFAULT '',
url_id INT NOT NULL,
CONSTRAINT fk_url_view FOREIGN KEY(url_id) REFERENCES url(id)
);
Creating Supabase project and tables
- From your Supabase dashboard , click
New project
. - Enter a
Name
for your Supabase project. - Enter a secure
Database Password
. - Select the
Region
you want. - Click Create new project.
- Open table editor.
- Click on
SQL editor
on sidebar. - Insert SQL table definition from the previous step.
- Click
Run
to create tables.
Designing the database
I always use interfaces for the database layer because of the following benefits:
- I can change a database simply by implementing a designed interface.
- It helps me to think and design a proper layer for the database.
- It helps to reduce dependencies between different parts of the codebase.
- It helps to write more modular and decoupled code.
- I can always implement a mock for my database layer by simply implementing a mock layer and use it in tests.
Implementing the Database
Supabase uses Postgres as their main RDBMS. Also, they use
PostgREST, a standalone web server that
turns your PostgreSQL database directly into a RESTful API. The structural
constraints and permissions in the database determine the API endpoints and
operations. The Supabase implementation of the database layer in
database/database.go
:
Designing the HTTP API
An URL shortener is a simple project so we need only two endpoints:
- Shorten URLs by passing a POST request to
/api/url
- Get created URLs by passing a GET request to
/api/url
endpoint
Here's the implementation for our API in api/api.go
:
Plugging everything together
Create a file in cmd/shorts/main.go
with the following content
Configuring Ory Oathkeeper
I'll use Ory Oathkeeper to implement zero trust network configuration. Ory Oathkeeper acts as a reverse proxy in this example, but it checks if the request is authenticated.
URLs that follow /u/hash
pattern should be available to anonymous users, and
the API should be available only for authenticated users.
Basic configuration for Ory Oathkeeper
Access rules for Ory Oathkeeper
Create a folder oathkeeper
to hold the Ory Oathkeeper access rules:
Authenticating users
You can follow the Ory Kratos Quickstart to add Ory Kratos to your project. You can inspect the docker-compose and the Ory Kratos configuration folder in the repository, but this basic configuration is based on the Quickstart guide.
Authentication middleware
We configured Ory Oathkeeper to act as identity and access proxy, and our configuration does two things:
- Check authentication and who the current session belongs by calling the
sessions/whoami
endpoint of Ory Kratos. - Get the identity.id from the previous step and modify the request by adding a
X-User
header with the user's identity id.
It means that we do not need to take any additional steps for our Go
application, and we can get the value from X-User
header and use it in our
application.
Next steps
We have implemented the basic URL shortener API and added authentication with Ory Kratos and a SQL backend powered by Supabase. Feel free to fork the example and play around with it.
I hope you enjoyed this tutorial and found it helpful. If you have any questions, check out the Ory community on Slack and GitHub.
Some ideas to improve the backend further:
- Enable 2FA for Ory Kratos
- Add support of AWS Lambda for backend to enable cost-effective hosting