Learning to Go: Part 5 - Structs & Querying the API

Learning to Go: Part 5 - Structs & Querying the API

Oh hi! This is a multi-part tutorial series, please make sure you reads the posts in order! You can find the list at the bottom of the page.

Structs

Our app can now accept search terms from a user, but what do we do with them? To make our search engine work, we’ll need to query a public API of dialog from Star Trek: The Next Generation episodes to search for matches. When I started writing this tutorial, to my suprirse and horror, I found that no such public API existed. So I created one :-)

In this stage of the tutorial we’ll learn how to query this API and access the results. When we access an API, we usually receive the results in the form of JSON. In Go, to store these results we need to create something called a struct. A struct is just another type of variable that we define, that contains other variables.

In order for everything to work nicely, the format of our struct has to exactly match the format of the JSON result we are expecting. Thankfully, most APIs are well-documented and you can use the excellent JSON-to-Go tool to create a Go struct for you based on a sample JSON output.

I’ve done this for you already and created the struct format we need to handle the API. Add this to main.go right after the imports section:

type Dialog []struct {
    Episodename string `json:"episodename"`
    Act         string `json:"act"`
    Scenenumber string `json:"scenenumber"`
    Texttype    string `json:"texttype"`
    Who         string `json:"who"`
    Text        string `json:"text"`
    Speech      string `json:"speech"`
    Released    string `json:"released"`
    Episode     string `json:"episode"`
    Imdbrating  string `json:"imdbrating"`
    Imdbid      string `json:"imdbid"`
    Season      string `json:"season"`
}

As you can see, this struct definition just gives us a way to create new variables of type Dialog, instead of say, string or int. Inside our struct are other variables, or fields, that we can access. Note how every field is adorned with its corresponding JSON key, which will be used by the JSON decoder later when we process the results.

We also create this as a []struct not just a struct making it a slice. In Go, you can think of a slice as a mutable array. It just means we can store multiple Dialog results.

Querying the API

Now we’ve defined our Structs, let’s update our searchHandler to query the API and store the results. Delete the following line from the function:

fmt.Println("Search Query is: ", searchKey)

From there, add the following code:

dialog := &Dialog{}

Here we create a new variable dialog, based on the Dialog struct we created a moment ago. This is a slightly odd and new way of creating a variable that we haven’t seen before.

The {} brackets make this what’s called a literal. This is just a way for us to literally define what should be inside our struct, if we want to. In this case we’re choosing to leave the literal blank, as we’ll populate it in a moment.

The & makes this a pointer to our new struct literal. For the purposes of this tutorial, don’t worry about pointers. You’ll learn about them soon enough, but there simply isn’t a concise way to describe them right now without throwing you completely off track :-)

Back to the code. It’s finally time to access that API!

endpoint := fmt.Sprintf("https://tngapi-awicwils6q-ew.a.run.app/?q=%s", url.QueryEscape(searchKey))
resp, err := http.Get(endpoint)
if err != nil {
    w.WriteHeader(http.StatusInternalServerError)
    return
}

Here we specify the API endpoint as a string and join it with our search term to create a URL to query (also using url.QueryEscape to make our search term URL-safe).

Then we use the Get function in the http library to make a request to this URL. We use the Go error handling pattern again, getting the response from the function as resp, any possible errors as err, then checking to make sure that err is nil before we carry on.

Just a quick note on this API: It’s running in one of my GCP projects, so it may be up, down, or rate-limited so it doesn’t cost me too much. Please don’t abuse it or I’ll have to take it down!

Next, we do a bit more error checking:

defer resp.Body.Close()
if resp.StatusCode != 200 {
    w.WriteHeader(http.StatusInternalServerError)
    return
}

First, we have to close the resp connection to our HTTP endpoint, but we use defer to literally defer this happening until the rest of our main function is complete. Then we check the actual response code of the API. Our function may have worked, but if there’s a problem with the remote API itself, we might have a problem decoding any JSON in the response, so we want to catch that now. Time to decode that JSON:

err = json.NewDecoder(resp.Body).Decode(&dialog)
if err != nil {
    w.WriteHeader(http.StatusInternalServerError)
    return
}

Here we use the NewDecoder function in the json library, sending it the Body of our response, then we call the Decode function on that. We pass a pointer to our dialog variable to the Decode function, which means that it can write directly to it - and we don’t actually need to return anything on this line. That’s why we just get the output as err to once again check for any errors along the way.

Remember when we created the Dialog struct, how every field is annotated with a JSON key? The Decoder can use these to neatly store the values in the JSON response in every corresponding field of the struct. This is why it is so important to match the response with the layout of your structs.

We’re nearly done! Next we’ll add something temporary to help us see what’s happening with our API request:

spew.Dump(search)

This uses an external library to pretty-print the entire search struct to our terminal when we run the program. We’ll use this now to show us that the structs and the query are working fine, before we move onto the next stage of the tutorial where we try to render the output properly. For now, we’ll just render the same template:

err = tpl.Execute(w, dialog)
if err != nil {
    w.WriteHeader(http.StatusInternalServerError)
}

Just like in the indexHandler, we call tpl.Execute to render a template, but this time we pass it the dialog variable as well. Remember this is our struct that contains the search term entered, plus the entire response from the API.

Before we can run this program, we first need to update our imports section. We no longer need fmt, but we’re adding a couple of new libraries:

import (
    "encoding/json"
    "html/template"
    "net/http"
    "net/url"

    "github.com/davecgh/go-spew/spew"
)

This shows you how easy it is to add external libraries to Go. But for this to work, you first need to “get” this library on your local system. Save your main.go file and then run this comamnd in your terminal:

go get -u github.com/davecgh/go-spew/spew

Once that’s done, run the program as normal with:

go run main.go

Go back to http://localhost:8080/ and enter a search term (try “Darmok”) and hit return. The page in your browser won’t change, but in your terminal you should get a dump of the entire struct with lots of results and dialog.

Great job! You’ve successfully queried an external API and stored its results in structs. In part 6 we’ll learn how to access struct fields in our HTML template so we can render our search results for the user.

Here’s the gist of main.go if you need to check yours for errors (it’s getting a bit too big to embed now!)