In this article, we’ll be talking about how to use Go and the gin gonic web framework to create a reverse proxy server. If you’re not familiar with what that is, it’s basically a middleman that sits between a client and one or more servers, forwarding requests and responses between them.
We’ll use the reverse proxy to call external APIs, limit the number of requests we make to them (to avoid hitting API limits), cache requests for faster retrieval, and make requests from the browser, including blocking and replacing content. All of these features can be super helpful in improving the functionality and performance of our application.
So, let’s dive in and see how to set up and configure our very own reverse proxy server!
Setup
First, you’ll need Go, and a good IDE. I think Visual Studio Code is currently the best IDE, and it works on all major platforms.
Next, initialize the project with the following commands. I’m using github.com/nathankjer/gin-proxy as my project path but you should use your own.
go mod init
go mod tidy
Creating the proxy server
Now let’s create a proxy server. Paste the code below (source: Sébastien Le Gall) into a file located at /proxy/proxy.go. I will be using Wikipedia as my example host, but you can use whatever you like.
package main
import (
"fmt"
"net/http"
"net/http/httputil"
"net/url"
"github.com/gin-gonic/gin"
)
func proxy(c *gin.Context) {
remote, err := url.Parse("https://en.wikipedia.org")
if err != nil {
fmt.Println(err)
}
proxy := httputil.NewSingleHostReverseProxy(remote)
proxy.Director = func(req *http.Request) {
req.Header = c.Request.Header
req.Host = remote.Host
req.URL.Scheme = remote.Scheme
req.URL.Host = remote.Host
req.URL.Path = c.Param("path")
req.URL.RawQuery = c.Request.URL.RawQuery
}
proxy.ServeHTTP(c.Writer, c.Request)
}
func main() {
r := gin.Default()
r.Any("/*path", proxy)
r.Run(":3001")
}
Let’s create a client that will use our proxy server. Paste the following within /client/client.go:
package main
import (
"fmt"
"io"
"net/http"
)
func main() {
res, err := http.Get("http://localhost:3001/w/api.php?action=parse&page=Proxy_server&format=json")
if err != nil {
fmt.Println(err.Error())
}
responseBody, err := io.ReadAll(res.Body)
if err != nil {
fmt.Println(err.Error())
}
fmt.Println(string(responseBody))
}
Now, run the proxy server with the following command. This will start our proxy server, which will be listening for requests on port 3001.
go run proxy/proxy.go
Next, run the following in a separate terminal window/tab:
go run client/client.go
You should get the following output:
There should be a 200 response on the proxy server:
Rate limiting
We can utilize gin’s built-in rate limiter by running the proxy server like so:
go run proxy/proxy.go -rps=100
This utilizes the leaky bucket method, ensuring that requests are made on average every 10 ms.
Creating a request server
Let’s add a request server that will forward our requests to the proxy server, or return a cached response for previously seen requests. This will allow us to never ask for the same information twice, which comes with the added benefit of 100 times faster response times.
Paste the following within server/server.go:
package main
import (
"encoding/json"
"fmt"
"io"
"net/http"
"time"
"crypto/md5"
"github.com/gin-gonic/gin"
"github.com/nathankjer/gin-proxy/db"
"github.com/nathankjer/gin-proxy/models"
)
func request(c *gin.Context) {
header, _ := json.Marshal(c.Request.Header)
body, _ := io.ReadAll(c.Request.Body)
request_id := fmt.Sprintf("%x", md5.Sum([]byte(c.Request.Method+c.Request.Host+c.Request.URL.String()+string(header)+string(body))))
var request models.Request
if err := db.Database.Where("id = ?", request_id).First(&request).Error; err != nil {
// Perform request using proxy
req, err := http.NewRequest(c.Request.Method, "http://localhost:3001"+c.Request.URL.Path+"?"+c.Request.URL.RawQuery, c.Request.Body)
if err != nil {
fmt.Println(err)
}
res, err := http.DefaultClient.Do(req)
if err != nil {
fmt.Println(err)
}
responseBody, err := io.ReadAll(res.Body)
if err != nil {
fmt.Println(err)
}
// Save request
request = models.Request{
Id: request_id,
Method: c.Request.Method,
Host: c.Request.Host,
Path: c.Request.URL.Path,
ResponseStatus: res.StatusCode,
ResponseBody: responseBody,
CreatedAt: time.Now(),
}
db.Database.Create(&request)
c.Data(res.StatusCode, "", responseBody)
} else {
c.Data(request.ResponseStatus, "", request.ResponseBody)
}
}
func main() {
db.Connect()
r := gin.Default()
r.Any("/*path", request)
r.Run(":3000")
}
We need a model to represent requests. Paste the following within models/requests.go:
package models
import "time"
type Request struct {
Id string `json:"id"`
Method string `json:"method"`
Host string `json:"host"`
Path string `json:"path"`
ResponseStatus int `json:"response_status"`
ResponseBody []byte `json:"response_body"`
CreatedAt time.Time `json:"expires_at"`
}
Next, let’s create an interface that will allow us to save requests to a PostgreSQL database. First, let’s create a database for our project:
echo "CREATE DATABASE ginproxy;" | psql --dbname postgres
Paste the following within db/db.go. Depending on your setup, you may need to add a password to the database config string:
package db
import (
"log"
"os"
"github.com/nathankjer/gin-proxy/models"
"gorm.io/driver/postgres"
gorm "gorm.io/gorm"
)
var Database *gorm.DB
func Connect() {
dbURL := "host=localhost user=postgres dbname=ginproxy port=5432"
database, err := gorm.Open(postgres.Open(dbURL), &gorm.Config{})
if err != nil {
panic("Failed to connect to database!")
}
Migrate(database)
Database = database
}
func Migrate(database *gorm.DB) {
database.AutoMigrate(&models.Request{})
}
Modify the port within client.go to 3000. Run proxy.go, server.go, and re-run client.go. The response for proxy and client, should be the same as before.
On server.go, you should get the following:
Run client.go again. You should now see a second request on server.go. The second request responded 100 times faster than the original request! As expected, no request was made to the proxy server.
Making requests from the browser
Not only can we use the proxy for APIs, but we can also use it to make requests from the browser! The animation shown previously is a good illustration. This can be used for reviewing traffic, blocking ads, and injecting content/styles.
We can use the code below to replace all instances of paragraph tags with one that is styled with a pink text color.
c.Data(res.StatusCode, "", []byte(strings.ReplaceAll(string(responseBody), "<p>", "<p style=\"color:pink;\">")))
Future improvements
The server does not fully support headers or cookies, or other nuances such as the user-agent. I designed it primarily to make requests to APIs but these are features that might be implemented in the future.
