Our task for the fourth week was to create a URL SHORTENER
BUB IT - A URL SHORTENER
DESIGN AND CASE STUDY BY IVY FOO
The only guide you'll ever need...
A url shortener is used to convert long, untrackable links into short, pleasant and memorable links.
A url shortener may seem unnecessary because the process might be too complicated and it's probably hard to wrap your head around it right now. The article provides a bit of a walkthrough. As developers, we'd be simplifying the process. We wouldn't want to walk you through a dark and thorny path.
Why create a url shortener? Why not?
- Short urls can be memorable.
- Long links can seem malicious.
Basically,
LONG URL:
cbsnews.com/news/missing-idaho-kids-jj-vall.. details-investigation-48-hours/
SHORT URL:
shorturl22.herokuapp.com/dG71svL
You will agree a short url looks better
The Data Capacity Model:
LONG URL : 2KB - 2048 characters
SHORT URL: 17B - 7 CHARACTERS
It's important to note that generally short urls use 7 characters which may be more sometimes.
Now, let's talk about how a short url is generated?
There are two algorithms we use to generate a short url:
Base 62
MD5 Hash
As you may know, the Base 62 provides Alphabetical characters both lowercase and uppercase and integers 0-9
MD5 Hash takes the long url and gives an output but MD5 gives a really lengthy output which we are trying to avoid.
It's much preferable to use the Base 62 algorithm.
Potential architecture to solve the problem
Let's talk about our UI that takes in the Long Url input, it calls the Short Url service which somehow generates the Short Url and stores in the database, the Short Url is called from the database and the user is redirected to the page
Covering how the URL service is generated
Now, let's say there are multiple instances of this service and all of them are getting requests. A collision is possible whereby two long urls have the same short urls. We could employ Redis. Redis makes sure to always return a unique number, it counts all the way from one to a billion or trillion. Each time it gets a request, it increments the counter and responds back. Now, we make sure they are getting unique numbers and unique numbers at Base 10 which can be converted to Base 62 and generate a Short Url.
There seems to be a small problem at this point. Yes, you guessed right! Redis could be getting lots and lots of requests A solution experts recommend is to keep multiple Redis that will give us high availability and it gives us better performance because there are multiple machines.
Let's look into another way to generate a short url without Redis. The idea is come up with a number and make sure there are no duplicates. The Short url service request a token range from a token service. The token service can be built with MYSQL because it runs at a very low scale. Let's say a request reaches this service. It takes the first number and converts it into base 62. It will request the next Token service and get the next range.
In a simple MYSQL, you get a record. When you get a request, you take the next token unassigned range and return it. Because it sits on top of MYSQL, a unique token is always generated.
Finally, it would be nice to get the kind of geography your people come from and ehat kind of hits you are getting. Let's say when a client sends us a request. We query the database, get the longer URL from the database based on the Short url and return it back to the client, which then redirects to the long url. This request will come with some a lot of attributes, so it will give us a header as to where this request has come in. So let's assume we posted the link on Facebook. It will have that information and information about the user. Based on the IP address, we will be able to come up with a geographical location.
The Mobile View of our URL shortener
Credits:
Ivy Foo
behance.net/gallery/146169531/URL-Shortener..
The Homepage:
The Homepage takes in the long url
The Result Page:
This page displays the result
The Recent Urls page:
This page displays a list of recent urls
The Detailed Stats page:
This page displays a detailed statistics. It displays the number of clicks, geographical location of the user and other minute details.
The backend team provided an API hosted on Heroku
A quick recap of what an API is: An application programming interface allows two programs to communicate on the web. APIs sit between an application and a web server and facilitate data transfer.
Let's make a quick illustration.
Imagine a full-sized colony of bees...
Bee one: 'Hey we need more nectar(data) to make our honey(app)'
Bee two quickly sends a request (HTTP request)
GET/nectar/pinkflowers.
Request: API call is initiated by the client application via a HTTP request
Receive: Our worker bee acts as an API, going to a flower(server) to collect nectar(data).
Response: The API transfers the required data back to the requesting application usually in JSON format.
Conclusion
The fourth week was eye opening!
Design Credit: Ivy Foo