HTTP SERVERS: BASIC LESSON
In this article, we are going to implement some of the basic APIs of the http module to serve static content on the web.
In the previous tutorial, we talked about the basic implementation of an HTTP server in Go and how routes work. If you haven’t read it, I would recommend going through it before continuing this article.
Creating a simple “Hello World!” HTTP Server in Go
On the web, we do not just serve processed HTML but files as well, like CSS, JavaScript, PDF, etc. Most of the time, these files are served as-is from the disk but sometimes, they need to be processed before delivering to the client.
In some cases, we serve the content of the directory and let the user decide which files they need. In this article, we are going to deal with some use case scenarios and implement some security measurements while doing so.
FileServer function
The http.FileServer() function provides the functionality to serve the entire file-system directory with indexes. Let’s have a look at its syntax.
func FileServer(root FileSystem) Handler
As you can see, it returns a Handler object which makes it a perfect candidate to be used as handler in ListenAndServe function or the Handle function. It takes an argument of FileSystem interface type.
type FileSystem interface {
Open(name string) (http.File, error)
}The root argument represents the file-system directory from which the content will be served. Luckily, we don’t have to create our own implementation of FileSystem interface.
http.Dir type
The Dir type may look like a function but it is an alias of string data type and it implements Open method defined by the FileSystem interface. We can call the Dir type like a function which is nothing but a type-casting syntax.
The string value passed to the type-casting syntax is a relative or an absolute directory on the native file-system.
var fs FileSystem = http.Dir("/tmp")💡 The http.Dir type does not exclude . prefixed files like .git or .htpasswd, so be careful what you are serving. Else, you can create your own implemention FileSystem interface. If we pass empty directory string “”, the Go will use the os.Executable() as the default directory to serve files from.
Serving a directory
I have created a simple temporary folder under my user directory to demonstrate this example. It looks as follows.
/Users/Uday.Hiwarale/tmp
├── .htpasswd
├── files
| └── test.pdf
├── main.js
├── page.html
└── style.css
Let’s write a simple Go HTTP server to serve /Users/Uday.Hiwarale/tmp directory using http.FileServer handler and http.Dir type.
From the above example, we can see that we did not implement any routes and all traffic goes through the fs request multiplexer. But if run this application and open the default URL in the browser, things are pretty simple.
If the request URL path is /, then FileServer returns the list of files and folders from the root directory. If the request URL path matches the path of a file or folder in the root directory, then that file or folder is returned.
💡 If you are wondering how the browser knew it was a PDF file, then you should inspect the Content-Type response header. FileServer is equipped with the functionality to set an appropriate Content-Type header based on the file type.
If the request URL path does not match a file or folder contained in the root directory, a 404 page not found message is returned with a 404 HTTP status.
Handling indexes
If you have some experience of working with Apache or Nginx server then you know what indexes are. An index is a list of files and folders contained in a directory. When a directory does not contain index.html file, these servers take the responsibility of returning HTML with a list of these files and folders.
This is exactly what our Go file server is doing. However, generally speaking, you do not want a user to see the content of the directory. In that case, you should take the responsibility of returning HTML with a directory index.
We can do that by placing index.html in every directory that can return an index. Let’s rename the page.html file to index.html contained in the root directory served by our HTTP server.
Now, if you access the root URL, our file server will return the index.html instead. This way, we have disabled access to the index of the root directory. However, the internal directories like files/ can still return an index if they don’t contain a index.html file.
From the example above, since index.html is a file inside root directory, we should be able to access it from /index.html URL path. However, Go will redirect any URL ending with /index.html to the ./ path (current directory of the file) with 301 Moved Permanantly status code.
Serving files on a route
In the previous article, we learned how to create routes and how to handle responses for individual routes. In a typical use-case scenario, we want a specific route to handle the static file serving part. Let’s create a /static route to serve the content of the /tmp directory.
Now that we have our program set, we can start the server and use /static route to access the files from /tmp directory as usual.
From the above result, we are not getting the desired result. For some reason, the Go HTTP server could not redirect the URL to the correct handler and the request is intercepted by the / handler.
This happens because http.Handle in comparison with http.HandleFunc needs a trailing / to successfully work. If we change /static route path to /static/, it should start working.
// handle `/static/` route using a Handler
http.Handle( "/static/", fs )
From the above result, we can see that the fs file-server handler is invoking but it’s not able to return the file. If you try any combination of URLs, the results will be the same. So what is causing this issue?
The problem is occurring because of the request URL. The http.FileServer serves the file by looking at the request URL. Since our request URL is /static/files/test.pdf, it will try to look for a file inside root directory with the path /tmp/static/files/test.pdf and it doesn’t exist.
Somehow, we need to remove /static part from the URL and invoke the fs handler to serve the file. This exactly what http.StripPrefix does.
func StripPrefix(prefix string, h Handler) Handler
Let’s modify our earlier example and implement the StripPrefix method.
// handle `/static/` route using a Handler
http.Handle( "/static/", http.StripPrefix( "/static", fs ) )
ServeFile function
The http package provides a ServeFile function to serve a file on the disk using its file path. This function has the below syntax.
func ServeFile(w http.ResponseWriter, r *http.Request, name string)
We are familiar with the first two arguments. The name argument is the path of the file on the disk. In all cases, an absolute path is recommended but it is a relative path, then things get a little weird.
Let’s create a simple HTTP server to serve some files using ServeFile function. We can reuse our earlier tmp/ directory for this use case.
From the program above, we are using a /pdf route path to get the test.pdf file and it works just fine.
However, there are some issues with the above program. First of all, the root directory path saved in TmpDir is separated by /. This is not platform dependent and other platforms like Windows can use \ as a separator.
To convert a platform-independent path to a platform-dependent path, we can use path/filepath package. The FromSlash function replaces the / with a valid separator like / in Unix systems and \ in the case of Windows.
var TmpDir = filepath.FromSlash("/Users/Uday.Hiwarale/tmp/")Second, we are joining the /files/test.pdf file using the + operator and it also contains /. We can use the filepath.Join() function to join two paths together with some sanitization (it removes extra slashes).
filepath.Join( TmpDir, "/files/test.pdf" )
Security measurements
As long as name argument of the ServeFile function is properly sanitized, we are fine. However, if a raw user input from URL is passed to this function, it can lead to massive security outbreaks.
For example, if the request URL contains .. path element, which loosely translates to move up the directory, we are granting access to the directory which could contain confidential files.
To resolve this issue, we can either reject requests containing .. elements or strip insecure path elements before serving a file. However, ServeFile function does have a built-in mechanism to deal with this situation, in which case, it rejects the request.
Handling index.html
As we have seen in the FileServer lesson, index.html contained in a directory acts as a directory index. But when we use /index.html in the request URL, it redirects to the closest / URL path and then serves it.
This happens also with the ServeFile function. If the request URL contains /index.html, then it will be redirected to ./ path (relative directory).
From the program above, we are serving index.html file from /index.html route. We have also added / path to handle any fallbacks.
As you can see, we were redirected to ./ path which is / for the index.html and we received the default response of / route.
To avoid such a situation, we can either stop using /index.html which is generally accepted over the web or we can use other mechanisms to serve a file’s content like http.ServeContent function.
Handling relative paths
If you are using ServeFile function and the name argument is an absolute path, then you don’t have anything to worry about the file’s delivery. But using a relative file path may lead to some problems.
We need to honor the fact that Go is a compiled language. Out of multiple program files located in multiple locations, we can create a binary executable. Any relative file paths used in these files hold no meaning in runtime.
os.Executable function
In most cases, we should try to use the absolute file path as much as possible. But in some cases, when a relative file path is important, then we should be using os.Executable function which returns the absolute path of the running executable file.
If you are shipping .exe file of your program and users have been instructed to put this file inside a directory that contains some static files your program depends on, then os.Executable() function comes in handy to get the directory of the running executable.
exePath, err := os.Executable() => /usr/share/program.exe
exeDir := filepath.Dir(exePath) => /usr/share/
os.Getwd function
In some cases, you need the directory of the terminal which has executed the Go program or executable file. In Bash, you may have used pwd command. The os.Getwd() returns the current working directory.
Beginners guide to serving files using HTTP servers in Go was originally published in RunGo on Medium, where people are continuing the conversation by highlighting and responding to this story.