Project

General

Profile

HTTP Client

Introduction

Getting network connectivity through something like the ESP8266 is pretty neat, but by itself doesn't really accomplish anything.

HTTP is the protocol that rules the web. It is used by your web browser to access this documentation wiki, as well as all sorts of applications that call upon web-based services.

Therefore, having the ability to communicate over HTTP with ease is a fundamental requirement for most IoT applications.

The Dooba SDK provides a http library which includes everything needed to accomplish just this.

This page will describe how to use this library to make HTTP requests. The server part of the library is documented here: HTTP Server.

Note: to keep things simple, the examples presented here will rely on the ESP8266 for internet connectivity, and assume they are being built for the Fiddle.

Async processing

To avoid blocking, the library is partly asynchronous.
Requests are sent right away (synchronous send), but then the response will be fetched asynchronously.
This allows the user to do something else while waiting for the response.

This implies that each request needs to be identified using a variable (of type struct http_client_req) when created, which will be re-used later on when checking the response.

Using the library

To use the library, simply add http as a dependency to your definition file (dfe.conf):

# Firmware Element Configuration
# ...
deps:
  - http

You'll want to include the following header files in your code:

  • http/http.h
  • http/client.h
  • http/client_util.h

Sending a simple request

Let's jump right in by looking at a complete example of how to send out a simple request.

First, here's a complete dfe.conf definition file for this example:

name: http_example_app
type: app
mmcu: atmega1284p
freq: 10000000L
deps:
  - scom
  - esp8266
  - http

Then, here's a simple application to send a basic GET request and display the results through the USB Serial Terminal (SCOM).

#include <avr/interrupt.h>
#include <scom/term.h>
#include <esp8266/esp8266.h>
#include <esp8266/net_transport.h>
#include <http/http.h>
#include <http/client.h>
#include <http/client_util.h>

// WiFi will need quite some space to store packets
#define wifi_buf_size            2048
uint8_t wifi_buf[wifi_buf_size];

// Request object
struct http_client_req r;

void main()
{
    // Initialize Serial Terminal
    scom_term_init(0);

    // Initialize WiFi
    esp8266_init(wifi_buf, wifi_buf_size);

    // Enable Interrupts
    sei();

    // Configure WiFi & Join Access-Point
    scom_term_printf("Connecting to WiFi...\n");
    esp8266_set_mux(1);
    esp8266_set_mode(ESP8266_MODE_STA);
    if(esp8266_join_ap("YourNetwork", "YourNetworkPassword"))
    {
        // Failed - Something went wrong
        scom_term_printf("Failed to connect to WiFi!\n");
        for(;;);
    }

    // Make GET Request
    scom_term_printf("Sending GET Request...\n");
    if(http_get(&r, &esp8266_net_transport, "example.com", 80, "/foo/bar", 0))
    {
        // Something went wrong... :(
        scom_term_printf("Failed to send request... :(\n");
        for(;;);
    }

    // Wait until we get a response
    while(http_req_finished(&r) == 0)        { esp8266_update(); }

    // Check Result
    if(http_req_failed(&r))
    {
        // Request Failed :(
        scom_term_printf("Failed to complete request... :(\n");
        for(;;);
    }

    // Print out result
    scom_term_printf("Got Response! Status: [%i] (%t)\n", r.res.status, http_res_status_txt(&r), r.res.status_txt_len);

    // ...
    for(;;);
}

Request methods

To send out requests with common HTTP verbs (POST / PUT / DELETE / etc...), the methods below are provided.

Note that "_data" variants are provided for POST, PUT and PATCH - these allow a custom payload (HTTP body) to be sent along with the request.

uint8_t http_get(struct http_client_req *r, struct net_transport *nt, char *host, uint16_t port, char *path_fmt, ...)
uint8_t http_post(struct http_client_req *r, struct net_transport *nt, char *host, uint16_t port, char *path_fmt, ...)
uint8_t http_post_data(struct http_client_req *r, struct net_transport *nt, char *host, uint16_t port, void *data, uint16_t size, char *path_fmt, ...)
uint8_t http_put(struct http_client_req *r, struct net_transport *nt, char *host, uint16_t port, char *path_fmt, ...)
uint8_t http_put_data(struct http_client_req *r, struct net_transport *nt, char *host, uint16_t port, void *data, uint16_t size, char *path_fmt, ...)
uint8_t http_patch(struct http_client_req *r, struct net_transport *nt, char *host, uint16_t port, char *path_fmt, ...)
uint8_t http_patch_data(struct http_client_req *r, struct net_transport *nt, char *host, uint16_t port, void *data, uint16_t size, char *path_fmt, ...)
uint8_t http_delete(struct http_client_req *r, struct net_transport *nt, char *host, uint16_t port, char *path_fmt, ...)
Arguments:

  • r -> Pointer to request (struct http_client_req)
  • nt -> Pointer to Network Transport
  • host -> Target host
  • port -> Target port
  • data -> Request body
  • size -> Request body size
  • path_fmt -> Format String for path

Return Value:
0 on success, anything else on error.

Note: when using these methods, the last argument should always be 0.
As you will see in the next section, these methods accept variable arguments which not only complete the path_fmt Format String, but can also be used to set request headers and even URL-Encoded Form Data.
Because of this, the last of the variable arguments must always be 0, even if no other argument is passed.

Setting request headers

Request headers can be set through the variable argument section of the methods presented above.

For each header that should be set, simply add one of HTTP_CLIENT_HEADER, HTTP_CLIENT_HEADER_N or HTTP_CLIENT_HEADER_F, followed by the header name and corresponding arguments:

  • HTTP_CLIENT_HEADER -> name, value (as null-terminated string)
  • HTTP_CLIENT_HEADER_N -> name, value, length
  • HTTP_CLIENT_HEADER_F -> name, value_fmt, ...

This allows great flexibility in setting the headers with only one method call.

The following example illustrates these usages:

http_get(&r, &esp8266_net_transport, "example.com", 80, "/foo/bar",
    HTTP_CLIENT_HEADER, "X-Simple-String", "this is a null-terminated string",
    HTTP_CLIENT_HEADER_N, "X-Fixed-Length-String", "Hello world!", 12,
    HTTP_CLIENT_HEADER_F, "X-Format-String", "This is more interesting: %i", 1234
    0);

Sending URL-Encoded Form Data

Just like headers, URL-Encoded Form Data can be provided through the variable arguments of the request methods presented above.

For each form field that should be set, simply add one of HTTP_CLIENT_FORMDATA, HTTP_CLIENT_FORMDATA_N or HTTP_CLIENT_FORMDATA_F, followed by the field name and corresponding arguments:

  • HTTP_CLIENT_FORMDATA -> name, value (as null-terminated string)
  • HTTP_CLIENT_FORMDATA_N -> name, value, length
  • HTTP_CLIENT_FORMDATA_F -> name, value_fmt, ...

The following example presents combined usage of headers and form fields in a POST request:

http_post(&r, &esp8266_net_transport, "example.com", 80, "/foo/bar",
    HTTP_CLIENT_HEADER_F, "X-Example-Header", "some integer: %i", 43,
    HTTP_CLIENT_FORMDATA, "first_field", "example value",
    HTTP_CLIENT_FORMDATA_N, "field_2", "fixed length value", strlen("fixed length value"),
    HTTP_CLIENT_FORMDATA_F, "3rd_field", "This is more interesting: %i", 1234,
    0);

Completing the request

Once a request has been issued, its completion can be checked using:

uint8_t http_req_finished(struct http_client_req *r)
Arguments:

  • r -> Pointer to request (struct http_client_req)

Return Value:
1 if request finished, 0 otherwise.

Of course, the underlying processes (esp8266 in these examples) need to run in order for the request to complete.
This is why in the first example at the beginning of this document does the following:

// Wait until we get a response
while(http_req_finished(&r) == 0)        { esp8266_update(); }

Handling the response

Note: all of the strings provided by the HTTP library are not NULL-terminated.
They are always accompanied by a _len variable (or similar) indicating their length.

Checking for success

Once the request is finished, the successful completion should be checked with:

uint8_t http_req_failed(struct http_client_req *r)
Arguments:

  • r -> Pointer to request (struct http_client_req)

Return Value:
1 if request failed, 0 otherwise.

If the request was a success, we can proceed with the response details.

HTTP status code

The HTTP Status (200 / 404 / 500 / ...) of the response can be accessed directly:

  • r.res.status (uint8_t) -> Status code as integer
  • char *http_res_status_txt(struct http_client_req *r) -> Status text ("OK" / "Not Found" / ...)
  • r.res.status_txt_len (uint8_t) -> Status text length
scom_term_printf("Got Response! Status: [%i] (%t)\n", r.res.status, http_res_status_txt(&r), r.res.status_txt_len);

Response headers

The response headers are available as an array:

  • r.res.headers (struct http_arg []) -> Array of struct http_arg objects
  • r.res.header_count (uint8_t) -> Number of headers in array

They can be easily searched using:

struct http_arg *http_res_head(struct http_client_req *r, char *name)
Arguments:

  • r -> Pointer to request (struct http_client_req)
  • name -> Header name

Return Value:
Pointer to header, 0 otherwise.

As mentioned above, headers are represented by struct http_arg objects.
They can be used with the following to obtain the name and value:

  • char *http_arg_name(struct http_client_req *r, struct http_arg *a) -> Pointer to name
  • char *http_arg_val(struct http_client_req *r, struct http_arg *a) -> Pointer to value
  • a->name_len (uint16_t) -> Name length
  • a->val_len (uint16_t) -> Value length

The example below prints out the Cookie header (if provided by the server):

struct http_arg *a;

// Find "Cookie" header
a = http_res_head(r, "Cookie");
if(a)   { scom_term_printf("Got Cookie: [%t]\n", http_arg_val(r, a), a->val_len); }
else    { scom_term_printf("No cookie provided by server!\n"); }

Response body

The full response body is accessible as:

  • char *http_res_body(struct http_client_req *r) -> Pointer to body
  • r.res.body_len (uint16_t) -> Body length