
import MDImage from '../../../components/MDImage.js';

# Function Inlining in Golang

## What is Inlining?

Inlining is a form of compiler optimisation where the compiler replaces a function call with the function's body to reduce the overhead of the function call. The following diagram crudely shows what this process looks like.

<MDImage src="/7/0_func_inlining.png" alt="Function inlining illustration" />

Let's take an example Go program.

```go
// main.go
package main

func main() {
	var a int = 10
	var b int = 20
	
	c := sum(a, b)
	print(c)
	
	d := difference(b, a)
	print(d)
}

func sum(a int, b int) int {
	return a + b
}

func difference(a int, b int) int {
	return a - b
}
```

Build the above program using the following command,
```sh
GOOS=darwin GOARCH=amd64 go build -o expinline_darwin_amd64
```

To understand inlining in practice, we will have to look at the assembly code generated by the above command and for that, we will have to disassemble the binary `expinline_darwin_amd64`.

Before we do that, let's look at the symbols (I am looking for the function names here),
```sh
$ objdump --syms expinline_darwin_amd64 | grep -E '_main.main|sum|difference'        
0000000001057600 l     F __TEXT,__text _main.main
0000000001014ba0 l     F __TEXT,__text _runtime.(*limiterEvent).consume
0000000001025000 l     F __TEXT,__text _runtime.(*pallocBits).summarize
000000000102c9c0 l     F __TEXT,__text _runtime.resumeG
```
You will see no symbols for the functions `sum` and `difference`. To understand why this is the case, let's dig deeper into the assembly code,
```sh
$ objdump expinline_darwin_amd64 --disassemble-symbols=_main.main
```
<MDImage src="/7/1_opt_and_inline_asm.jpeg" alt="" />

Surprisingly, I see no function calls (`callq`) to `sum` or `difference`. It looks as if the compiler was smart enough to reduce the whole program to calling the print function on two absolute constants `30` (`a`+`b`) and `10` (`b`-`a`) denoted by `movl $30, %eax` and `movl $10, %eax` respectively, before the call to `print`.

Let's turn off this compiler optimisation to preserve calculation logic and reevaluate the results.
```sh
$ GOOS=darwin GOARCH=amd64 go build -gcflags='-N' -o expinline_darwin_amd64_no_op
```

And examine the `_main.main` symbol again,
```sh
$ objdump expinline_darwin_amd64_no_op --disassemble-symbols=_main.main
```
<MDImage src="/7/2_no_opt_asm.jpeg" alt="" />

Now you can observe that there is addition logic between the offsets `105762a` and `105763c`  and subtraction logic between the offsets `1057665` and `105767c`. Again, there are no function calls (`callq`) to `sum` or `difference`, indicating that they have been inlined into the caller `main()`.

Along with turning off the optimisation, `inlining` can also be turned off.
```sh
$ GOOS=darwin GOARCH=amd64 go build -gcflags='-N -l' -o expinline_darwin_amd64_no_op_no_inline
```
If you examine the symbols now, you can see the missing functions as well:
```sh
$ objdump --syms expinline_darwin_amd64_no_op_no_inline | grep -E '_main.main|sum|difference'
00000000010576e0 l     F __TEXT,__text _main.difference
0000000001057600 l     F __TEXT,__text _main.main
00000000010576a0 l     F __TEXT,__text _main.sum
0000000001014ba0 l     F __TEXT,__text _runtime.(*limiterEvent).consume
0000000001025000 l     F __TEXT,__text _runtime.(*pallocBits).summarize
000000000102c9c0 l     F __TEXT,__text _runtime.resumeG
```
And here's the disassembled code,
<MDImage src="/7/3_no_opt_no_inline_asm.jpeg" alt="" />

## Why inline?

### Improves performance by minimising the function call overheads
These overheads include stack manipulation, CPU register loading/unloading, branching, handling return values, etc.

Let's perform a quick benchmark test to gauge the gain in performance. The following bench function calls the `sum` and `difference` functions,
```go
// main_bench_test.go
package main

import "testing"

var Result int

func BenchmarkSpecialBinomialProduct(b *testing.B) {
	var r int
	for i := 0; i < b.N; i++ {
		r = sum(i, i+7) * difference(i+7, i)
	}
	Result = r
}
```

Run it with all the default optimisations turned on,
```sh
$ go test -run='^$' -bench=. -benchmem -benchtime=10s -count=10 > bench_with_inlining.txt
```

Now add the compiler directive `//go:noinline` just above the function definitions of `sum` and `difference` and rerun the bench,
```sh
$ go test -run='^$' -bench=. -benchmem -benchtime=10s -count=10 > bench_with_no_inlining.txt
```

Finally, compare the results from the benchmark above,
```sh
$ benchstat {bench_with_no_inlining,bench_with_inlining}.txt
```
<MDImage src="/7/4_benchstat.png" alt="" />

Albeit a contrived example, you can see a performance improvement of ~78%.

Two things to note about inlining.
* It is a trade-off between space (increased binary size) and time (performance). This is because the function's body is copied to everywhere it is called.
* It yields diminishing returns as the functions get larger or more complex because the call overhead to these functions becomes relatively negligible.

### Augments escape analysis

Escape analysis is a compiler optimisation technique that determines if a value can be on the stack frame for the function constructing it or if the value must “escape” to the heap.

Let's consider the following example,
``` go
// escape/main.go
package main

type Point struct {
	x, y int
}

func NewPoint(x, y int) *Point {
	return &Point{x, y} // 8:9
}

func main() {
	p1 := NewPoint(10, 20) // 12:16
	p2 := NewPoint(30, 40) // 13:16
	
	print(p1.x, p2.y)
}
```

Build the above program with the `-m` compiler flag to print optimisation decisions,
```sh
$ go build -gcflags='-m=1' escape/main.go         
# command-line-arguments
escape/main.go:7:6: can inline NewPoint
escape/main.go:11:6: can inline main
escape/main.go:12:16: inlining call to NewPoint
escape/main.go:13:16: inlining call to NewPoint
escape/main.go:8:9: &Point{...} escapes to heap
escape/main.go:12:16: &Point{...} does not escape
escape/main.go:13:16: &Point{...} does not escape
```

Since `NewPoint` is expanded inside the `main` function, the lifetimes of `Point{10, 20}` and `Point{30, 40}` are bound within the `main`'s scope. 

Now build with inlining turned off,
```sh
$ go build -gcflags='-m=1 -l' escape/main.go
# command-line-arguments
escape/main.go:8:9: &Point{...} escapes to heap
```

Each call to `NewPoint(..)` will result in a `Point` object allocated to the heap memory.

## How does the compiler do that?

Go's compiler uses a combination of budget and cost to determine whether to inline a function. Every function has a cost compared against a budget (threshold). 

A function's cost is calculated using the complexity of the statements within the function body (one unit per node in the AST). And the budget changes from one release (40 in Go 1.6 | [src/cmd/compile/internal/gc/inl.go](https://github.com/golang/go/blob/7bc40ffb05d8813bf9b41a331b45d37216f9e747/src/cmd/compile/internal/gc/inl.go#L14)) to another (80 | [src/cmd/compile/internal/inline/inl.go](https://github.com/golang/go/blob/4be921d888d3a68c51e38d4c615a4438c7b2cb30/src/cmd/compile/internal/inline/inl.go#L47)).

Let's take the following example,
<MDImage src="/7/5_cost_budget_midstack.jpeg" alt="" />

Figures `a`, `b` and `c` represent the same program, each with slight modifications (green highlight) to demonstrate cost allocation and mid-stack inlining.

Let's build the program in Figure `a` (output condensed for relevance),
```sh
$ GOOS=darwin GOARCH=amd64 go build -gcflags='-m=2' midstack/main.go                      
# command-line-arguments
midstack/main.go:30:6: can inline Sum with cost 4 as: func(int, int) int { return a + b }
midstack/main.go:23:6: can inline MidPoint with cost 28 as: func(Point, Point) Point { return Point{...} }
midstack/main.go:25:9: inlining call to Sum
midstack/main.go:26:9: inlining call to Sum
midstack/main.go:18:6: cannot inline PrintPoint: function too complex: cost 89 exceeds budget 80
midstack/main.go:20:13: inlining call to fmt.Println
midstack/main.go:10:6: cannot inline main: function too complex: cost 114 exceeds budget 80
midstack/main.go:13:26: inlining call to MidPoint
midstack/main.go:13:26: inlining call to Sum
midstack/main.go:13:26: inlining call to Sum
```

You can see that each function has a cost assigned to it, which is compared against the budget (`80`) to decide whether the function can be inlined. The functions `MidPoint` (cost: `28`) and `Sum` (cost: `4`) are inline-able, whereas `PrintPoint` (cost: `89`) and `main` (cost: `114`) are not.

I will also use this example to showcase the two forms of inlining,
1. Leaf inlining
	- Earlier, only functions that did not call other functions (bottom of the call stack) were eligible for inlining. Once a leaf function is inlined, the resulting code can be inlined into its caller, a recursive process.
	- For example, `Sum` (a leaf function) gets inlined into `MidPoint`. Next, `MidPoint` (a leaf function now) gets inlined into `main`.
2. Mid-stack inlining
	- Now, non-leaf functions are also eligible for inlining.
	- For example, in Figure `c`, the function `Sum` is marked with the directive `//go:noinline`, making it non-eligible for inlining. Even though `MidPoint` calls `Sum`, it is allocated a cost of `76`, making it eligible for inlining.

## References
1. [Inlining optimisations in Go - Dave Cheney](https://dave.cheney.net/2020/04/25/inlining-optimisations-in-go#easy-footnote-5-4053)
2. [Mid-stack inlining in Go - Dave Cheney](https://dave.cheney.net/2020/05/02/mid-stack-inlining-in-go#easy-footnote-bottom-1-4076)
3. [Inlining - bright victories and hidden defeats](/7/6_inlining_2019_bright_victories_and_hidden_defeats.pdf)
4. [Five things that make Go fast - Dave Cheney](https://dave.cheney.net/2014/06/07/five-things-that-make-go-fast)
5. [Mid-stack inlining in the Go compiler - David Lazar](/7/7_talk_Mid-stack_inlining_in_the_Go_compiler.pdf)


Function Inlining in Golang


# Go RPCs (Remote Procedure Calls)

## Introduction

Go's `rpc` package enables communication across multiple machines, sort of like inter-process communication via funtion/method/procedure calls.

As the Go documentation puts it:
> Package `rpc` provides access to the exported methods of an object across a network or other I/O connection. A server registers an object, making it visible as a service with the name of the type of the object.

The object that is registered with the `rpc` server can be either primitive/builtin data types (`int`, `string` etc.) or user-defined ones (such as `struct`s). I would usually prefer a `struct`.

Under the hood, by default, Go uses `gob` for serializing and deserializing the remote procedure calls. This is configurable, that is, you can plug-in `JSON` or other custom codecs.

## Prerequisites

The methods which define actions on the registered object must adhere to a specific function signature. The methods,
- and its type(s) must be exported (uppercased)
- have two arguments, both exported (or builtin) types.
- their second argument is a pointer.
- must have one return type, `error`.

Essentially the method signature looks like the following:
```go
func (t *T) MethodName(argType T1, replyType *T2) error
```

## Implementation

Let's look at a dummy blog server and a client interacting with each other via `rpc`.
### a. Types

It's a good idea to wrap the object, along with the methods and types, into a package of its own. This makes it easier for both the server and client implementations to agree on common type definitions.

In the following code, `Blog` will be registered with the `rpc` server.
```go
type Blog struct {
	posts      map[int]Post
	lastPostID int
	sync.Mutex
}

type Post struct {
	ID    int
	Title string
	Body  string
}

// NewBlog is required because, even though Blog is exported its fields are not.
// The fields are internal to the package.
func NewBlog() *Blog {
	return &Blog{
		posts: make(map[int]Post),
	}
}
```
The function `NewBlog() *Blog` is necessary since we aren't exporting its fields: `posts` and `lastPostID`.

### b. Methods
`Blog` will have two methods: `AddPost` and `GetPostByID`. Notice the function signature.
```go
func (b *Blog) AddPost(payload, reply *Post) error {
	b.Lock()
	defer b.Unlock()

	if payload.Title == "" || payload.Body == "" {
		return errors.New("Title and Body must not be empty")
	}

	b.lastPostID++

	*reply = Post{ID: b.lastPostID, Title: payload.Title, Body: payload.Body}
	b.posts[reply.ID] = *reply

	return nil
}

func (b *Blog) GetPostByID(payload int, reply *Post) error {
	b.Lock()
	defer b.Unlock()

	*reply = b.posts[payload]

	return nil
}
```

Each method takes two arguments, `payload` and `reply` (you can name them anything). It receives the input from the client in the first parameter. The second argument (a pointer) is used for sending a response back to the client.

### c. Server
The server implementation is fairly simple:
```go
// Fetch an instance of the object
blog := types.NewBlog()

rpc.Register(blog) // Register the instance with the rpc
rpc.HandleHTTP() // Configure the rpc to serve over HTTP

err := http.ListenAndServe(":3000", nil)
if err != nil {
	log.Fatalln("Error starting the RPC server", err)
}
```

### d. Client

Dial the server using `rpc.DialHTTP`:
```go
client, err := rpc.DialHTTP("tcp", ":3000")
if err != nil {
	log.Fatalln("Error creating the RPC client", err)
}
```

And make the remote procedure calls using `client.Call`. It takes three arguments:
1. The method name of the form: `<T>.<MethodName>`
2. Input params
3. A pointer to receive the response from the server
```go
// Create a post
var post types.Post

// Create posts
err = client.Call("Blog.AddPost", &types.Post{Title: "post 1", Body: "Hello, world!"}, &post)
if err != nil {
	log.Fatalln("Error creating post", err)
}
log.Printf("[AddPost] ID: %d | Title: %s | Body: %s\n", post.ID, post.Title, post.Body)

// Fetch a post by ID
err = client.Call("Blog.GetPostByID", 3, &post)
if err != nil {
	log.Fatalln("Error creating post", err)
}
log.Printf("[GetPostByID] ID: %d | Title: %s | Body: %s\n", post.ID, post.Title, post.Body)
```

The entire code can be found on [Github](https://github.com/bensooraj/blog-code-resource-repo/tree/main/borneo).

## Some thoughts

In our implementation, the initial handshake between the `rpc` client and server are negotiated over HTTP. Thereafter, the HTTP server acts like a proxy or a conduit between the two, a concept known as [HTTP tunneling](https://en.wikipedia.org/wiki/HTTP_tunnel). I believe this can be customized.

It can be difficult to standardize (to follow common semantics) RPC implementations. This gap is what projects like `gRPC`, `dRPC` etc. attepmt to solve.

It should be obvious by now that you can register only one object with a given name with the `rpc` server.

Further reading:
1. [godoc: Package `rpc`](https://golang.org/pkg/net/rpc/)
2. [gRPC: A high performance, open source universal RPC framework](https://grpc.io/)
3. [dRPC: A lightweight, drop-in replacement for gRPC](https://github.com/storj/drpc)

---

Note: _This article is not an in-depth tutorial or treatment of Golang's syntax, semantics, design or implementation, but a journal of my learnings._

Go RPCs (Remote Procedure Calls)


# The empty interface and the empty struct in golang

An `interface` is a collection or set of method declarations. A data type implements or satisfies an `interface` if it at least defines the methods declared by the `interface`.

## Empty interface

An empty interface `interface{}` has zero methods. So, in essence, any data type implements or satisfies an empty interface. Let's take the following example:

```go
type Container []interface{}

func (c *Container) Put(elem interface{}) {
    *c = append(*c, elem)
    switch elem.(type) {
    case int:
        fmt.Println("Put ", elem, " of type int")
    case string:
        fmt.Println("Put ", elem, " of type string")
    default:
        fmt.Println("Put ", elem, " of type unknown")
    }
}

func (c *Container) Pop() interface{} {
    elem := (*c)[0]
    *c = (*c)[1:]
    return elem
}

func main() {
    allContainer := &Container{}

    allContainer.Put("Hello")
    allContainer.Put(213)
    allContainer.Put(123.321)

    fmt.Printf("allContainer: %+v\n\n", allContainer)
}
```

Output:

```sh
$ go run main.go
Put  Hello  of type string
Put  213  of type int
Put  123.321  of type unknown
allContainer: &[Hello 213 123.321]
```

In the above example, the data type `Container` is a slice of empty interfaces with two methods defined on it, `Put()` and `Pop()`. It loosely resembles a _First In, Last Out_ stack.

The `Put()` method takes an `interface{}` as the input argument and appends it to the slice of `interface{}`s. This means that it can accept a string, an integer, a float or any other simple and composite data type as its input argument. You can access the underlying type of `elem`, an `interface{}`, using `elem.(type)`.

This approach does have its disadvantages though. One being that you should implement type assertions, checks or type specific logics to avoid any surprises during runtime.

## Empty struct

To quote the Golang spec:

> A struct is a sequence of named elements, called fields, each of which has a name and a type.

Let's take an example

```go
// A struct with 5 fields.
struct {
    x, y int
    u float32
    A *[]int
    F func()
}
```

An empty struct is a struct data type with zero fields:

```go
// Named type
type EmptyStruct struct{}

// Variable declaration
var es struct{}

// Or use it directly!
struct{}{}
```

The size or width of a `struct` is defined as the sum of its constituent types. An empty `struct`, since it has no fields within it, has a size or width of zero. Zero bytes!

Just like regular structs you can define methods on empty structs as well. Sort of like a zero sized container for methods:

```go
type EmptyStruct struct{}

func (es *EmptyStruct) WhoAmI() {
    fmt.Println("I am an empty struct!")
}
```

Empty structs find its best use case in channel signalling. Many a mere mortals such as me have used booleans or integers to notify an event over a channel.

The following example prints the current time at every 500 milliseconds and times out after 3 seconds,

```go
func printFor3Seconds(doneChannel chan struct{}) {
    ticker := time.NewTicker(500 * time.Millisecond)
    timeout := time.After(3 * time.Second)

    for {
        select {
        case t := <-ticker.C:
            fmt.Printf("Tik tik: %v\n", t)
        case <-timeout:
            fmt.Printf("Timeout at: %v\n", time.Now())
            doneChannel <- struct{}{}
            return
        }
    }
}

func main() {
    doneChannel := make(chan struct{}, 1)

    go printFor3Seconds(doneChannel)

    <-doneChannel
}

// Output
// $ go run 1.go
// Tik tik: 2020-09-16 08:25:36.750631 +0530 IST m=+0.501680080
// Tik tik: 2020-09-16 08:25:37.250303 +0530 IST m=+1.001337269
// Tik tik: 2020-09-16 08:25:37.750696 +0530 IST m=+1.501714888
// Tik tik: 2020-09-16 08:25:38.253708 +0530 IST m=+2.004711715
// Tik tik: 2020-09-16 08:25:38.752804 +0530 IST m=+2.503793180
// Tik tik: 2020-09-16 08:25:39.251941 +0530 IST m=+3.002914584
// Timeout at: 2020-09-16 08:25:39.251999 +0530 IST m=+3.002972846
```

Instead of `doneChannel <- 0` or `doneChannel <- true`, I am using `doneChannel <- struct{}{}` for channel signalling. Using booleans or integers for channel signalling involves memory allocation, copying over the element etc.

We can use empty structs for simulating sets as well,

```go
intSet := make(map[int]struct{})
empty := struct{}{}

// We are setting empty as the value for 1, because the
// value is meaningless
intSet[1] = empty
if _, ok := intSet[1]; ok {
    fmt.Println("1 is in the set")
}

// 2 is not one of intSet's keys
if _, ok := intSet[2]; !ok {
    fmt.Println("2 is not in the set")
}
```

## Resources:

1. [The Go Programming Language Specification](https://golang.org/ref/spec#Struct_types)
2. [The empty struct - Dave Cheney](https://dave.cheney.net/2014/03/25/the-empty-struct)

---

Note: _This article is not an in-depth tutorial or treatment of Golang's syntax, semantics, design or implementation, but a journal of my learnings._

The empty interface and the empty struct in golang


import MDImage from '../../../components/MDImage.js';

# Sending application logs from EKS to Datadog

Datadog has been part of our stack for quite sometime now, where the `datadog-agent` installed on the EC2 instances (baked into the image) tail log files to which the application services write to.

A recent project required me to send the application logs deployed on kubernetes to Datadog. This required a different approach which I would be sharing today. A few pre-requisites before I begin:

1. An up and running kubernetes cluster. I use [`eksctl`](https://eksctl.io/) to quickly spin up a k8s cluster on AWS EKS.
2. A [Datadog](https://www.datadoghq.com/) account
3. Helm v3 installed (v3 doesn't require you to setup Tiller (Helm's server component) on your kubernetes cluster)

## Pre-requisites

To set the context, I have a 3-node cluster set up:
```sh
$ kubectl get nodes
NAME                                        STATUS   ROLES    AGE   VERSION
ip-10-0-0-34.ap-south-1.compute.internal    Ready    <none>   9d    v1.16.13-eks-2ba888
ip-10-0-1-144.ap-south-1.compute.internal   Ready    <none>   9d    v1.16.13-eks-2ba888
ip-10-0-2-79.ap-south-1.compute.internal    Ready    <none>   9d    v1.16.13-eks-2ba888
``` 

Add and update the Datadog chart repository,
```sh
# Ensure that you are running helm v3
$ helm version --short
v3.3.1+g249e521

# Add and update the Datadog and stable chart repositories
$ helm repo add datadog https://helm.datadoghq.com
"datadog" has been added to your repositories

$ helm repo add stable https://kubernetes-charts.storage.googleapis.com/
"stable" has been added to your repositories

$ helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "stable" chart repository
Update Complete. ⎈Happy Helming!⎈

$ helm repo list
NAME   	URL                                              
stable 	https://kubernetes-charts.storage.googleapis.com/
datadog	https://helm.datadoghq.com        

# Search for datadog chart. We will be using datadog/datadog
$ helm search repo datadog
NAME           	CHART VERSION	APP VERSION	DESCRIPTION             
datadog/datadog	2.4.13       	7          	Datadog Agent           
stable/datadog 	2.3.42       	7          	DEPRECATED Datadog Agent
```

Clone the git repository `bensooraj/node-eks-datadog-integration`:
```sh
$ git clone https://github.com/bensooraj/node-eks-datadog-integration.git

# A quick overview overview of the project structure
$ tree -L 2
.
├── Makefile
├── README.md
├── coverage
├── datadog
│   └── values.yml
├── docker
│   └── Dockerfile
├── kubernetes
│   ├── deployment.yaml
│   └── service.yaml
├── node_modules
│   ├── @dabh
│   ....
│   ...
│   ..
│   ├── winston
│   └── winston-transport
├── package-lock.json
├── package.json
└── server.js

83 directories, 9 files
```

Check [datadog-values.yaml](https://github.com/DataDog/helm-charts/blob/master/charts/datadog/values.yaml) for the latest YAML file.

## Deploy the application

We will be deploying an ultra simple Node.js API which logs out JSON formatted log messages whenever it receives a `GET` request:
```js
// server.js
const express = require('express');
const winston = require('winston');

// Constants
const PORT = 8080;
const HOST = '0.0.0.0';

const app = express();

const logger = winston.createLogger({
    transports: [
        new winston.transports.Console({
            handleExceptions: true,
        })
    ],
    level: 'debug',
    exitOnError: false,
    format: winston.format.combine(
        winston.format.splat(),
        winston.format.timestamp(),
        winston.format.json()
    ),
});

app.get('/', (req, res) => {
    logger.silly("This is a silly message", { url: req.url, environment: "test" });
    logger.debug("This is a debug message", { url: req.url, environment: "test" });
    logger.info("This is an info message", { url: req.url, environment: "test" });

    res.json({
        response_message: 'Hello World'
    });
});

app.listen(PORT, HOST);
logger.info("Running on http://${HOST}:${PORT}", { host: HOST, port: PORT, environment: "test" });
```

Relevant sections from the deployment YAML file,
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: eks-datadog-demo-node-app
  annotations:
    ad.datadoghq.com/eks-datadog-demo-node-app.logs: '[{"source":"nodejs","service":"eks-datadog-demo-node-app"}]'  
  labels:
    tags.datadoghq.com/env: "test"
    tags.datadoghq.com/service: "eks-datadog-demo-node-app"
    tags.datadoghq.com/version: "v1.0.1"
spec:
  # # #
  # #
  #
  template:
    metadata:
      annotations:
        ad.datadoghq.com/eks-datadog-demo-node-app.logs: '[{"source":"nodejs","service":"eks-datadog-demo-node-app"}]'
      labels:
        app: eks-datadog-demo-node-app
        tags.datadoghq.com/env: "test"
        tags.datadoghq.com/service: "eks-datadog-demo-node-app"
		tags.datadoghq.com/version: "v1.0.1"
	spec:
      containers:
        - name: eks-datadog-demo-node-app
  # # #
  # #
  #
```
Pay close attention to the `labels` and `annotations`. These are especially useful if you are planning to collect `traces` and application `metrics`, which are outside the scope of this article.

It's worth mentioning that you can exclude or include logs from deployments and/or pods by configuring the settings in the [`annotations`](https://docs.datadoghq.com/agent/logs/advanced_log_collection/?tab=kubernetes#exclude-at-match). For example, to exclude all logs from the above deployment configure the `log_processing_rules` in the `annotations` section as follows:
```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: eks-datadog-demo-node-app
  annotations:
    ad.datadoghq.com/eks-datadog-demo-node-app.logs: >-
      [{
        "source": "nodejs",
        "service": "eks-datadog-demo-node-app",
        "log_processing_rules": [{
          "type": "exclude_at_match",
          "name": "exclude_this_deployment",
          "pattern":"\.*"
        }]
      }]  
  labels:
    tags.datadoghq.com/env: "test"
    tags.datadoghq.com/service: "eks-datadog-demo-node-app"
    tags.datadoghq.com/version: "v1.0.1"
spec:
  # # #
  # #
  #
```

Deploy the API and exopse it as a service,
```sh
# Deployment
$ kubectl apply -f kubernetes/deployment.yaml

# Service
$ kubectl apply -f kubernetes/service.yaml

# Verify that the API is up and running
$ kubectl get deployment,svc,po
NAME                                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/eks-datadog-demo-node-app          2/2     2            2           30h

NAME                                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/eks-datadog-demo-node-app          ClusterIP   172.20.205.150   <none>        8080/TCP   30h

NAME                                                    READY   STATUS    RESTARTS   AGE
pod/eks-datadog-demo-node-app-7bf7cf764f-gpqvf          1/1     Running   0          29h
pod/eks-datadog-demo-node-app-7bf7cf764f-spb7m          1/1     Running   0          29h
```

## Deploy the Datadog agent

Open the file `datadog/values.yml` and ensure that `datadog.logs.enabled` and `datadog.logs.containerCollectAll` are set to `true`.
```yaml
datadog:
  ## @param logs - object - required
  ## Enable logs agent and provide custom configs
  #
  logs:
    ## @param enabled - boolean - optional - default: false
    ## Enables this to activate Datadog Agent log collection.
    #
    enabled: true
    ## @param containerCollectAll - boolean - optional - default: false
    ## Enable this to allow log collection for all containers.
    #
    containerCollectAll: true
```

Deploy the datadog-agent:
```sh
$ helm install datadog-agent -f datadog/values.yml  --set datadog.apiKey=<DATADOG_API_KEY> datadog/datadog

# Helm will deploy one datadog-agent pod per node
$ kubectl get deployment,svc,po
NAME                                               READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/datadog-agent-kube-state-metrics   1/1     1            1           29h

NAME                                       TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)    AGE
service/datadog-agent-kube-state-metrics   ClusterIP   172.20.129.97    <none>        8080/TCP   29h

NAME                                                    READY   STATUS    RESTARTS   AGE
pod/datadog-agent-b4mtv                                 2/2     Running   0          29h
pod/datadog-agent-f6jn7                                 2/2     Running   0          29h
pod/datadog-agent-zvp6p                                 2/2     Running   0          29h
```

Note that even though the agent is deployed in the `default` namespace, it can collect logs and metrics from deployments/pods from all namespaces.

## O Logs! Where art thou?

Now we have the API service (for generating logs) and the Datadog agent (for streaming the logs to Datadog) all set! I assume this is what happens behind the scenes:

1. The API service writes logs to **`stdout`** and **`stderr`**
2. The **`kubelet`** writes these logs to **`/var/log/pods/<namespace>_<pod_name>_<pod_id>/<container_name>/<num>.log`** on the node machine
3. The **`datadog-agent`**s running on each node tails these log files and streams them to the Datadog servers

Anyways, let's generate some logs:
```sh
# Port forward to access the API service from your local machine
$ kubectl port-forward service/eks-datadog-demo-node-app 8080:8080
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080

# Generate some logs using curl or ab
$ ab -n 50 -c 2 -l http://localhost:8080/

# Verify that the logs are generated
j$ kubectl logs -l app=eks-datadog-demo-node-app --since=100m
{"url":"/","environment":"test","level":"debug","message":"This is a debug message","timestamp":"2020-09-04T19:35:52.126Z"}
{"url":"/","environment":"test","level":"info","message":"This is an info message","timestamp":"2020-09-04T19:35:53.308Z"}
....
...
{"url":"/","environment":"test","level":"debug","message":"This is a debug message","timestamp":"2020-09-04T19:35:53.310Z"}
{"url":"/","environment":"test","level":"info","message":"This is an info message","timestamp":"2020-09-04T19:35:53.310Z"}
```

Navigate to `Log Explorer` in Datadog to see the logs that we generated above

<MDImage src="/5/2_dd_ui_overview.png" alt="Datadog UI overview" />

Click on any one of the log records to view more details

<MDImage src="/5/2_dd_log_detail_view.png" alt="Datadog log record detail view" />

On the left pane, you can filter the logs by namespace, pods etc. as well!

<MDImage src="/5/2_filter_ui_ns.png" alt="Datadog log record detail view UI" />

That's it for now!

I hope this helped and if you have any feedback for me, please let me know :smile:.

Further reading:
1. [Logging Architecture](https://kubernetes.io/docs/concepts/cluster-administration/logging/)
2. [Datadog kubernetes log collection](https://docs.datadoghq.com/agent/kubernetes/log/?tab=daemonset#log-collection)
3. [Datadog agent as a DaemonSet (recommended)](https://docs.datadoghq.com/agent/kubernetes/?tab=daemonset)
---

Note: _This article is not an in-depth tutorial on implemending logging for applications running on kubernetes, but a journal of my learnings._

Sending application logs from EKS to Datadog


# Struct aggregation and embedding in golang

A **struct**,
- is a collection of fields of various types, incuding other structs!
- can be used to create user-defined data types
- can have methods defined on them
- its fields and methods are accessed using the **dot** notation

For example, `Point` is a struct which has,

- two fields `X` and `Y` of type `float64`
- has two methods `String` and `MakesNoSense`

```go
// structs.go
// Struct definition
type Point struct {
	X, Y float64
}

// String overrides how an entity of type `Point` 
// is printed (for example, to `stdout`)
func (p Point) String() string {
	return fmt.Sprintf("Point {%f, %f}", p.X, p.Y)
}

// MakesNoSense is just a dummy method
func (p Point) MakesNoSense() {
	fmt.Printf("Point {%f, %f} is just a point!\n", p.X, p.Y)
	return
}

func main() {
	var pt Point
	pt.X = 1.123 // 
	pt.Y = 2.123 // 

    fmt.Println(pt)
    pt.MakesNoSense()
}
```

results in,
```sh
$ go run structs.go
Point {1.123000, 2.123000}
Point {1.123000, 2.123000} is just a point!
```

## Nested Structs

Yes, you can define one struct inside another. There are two ways to go about it: **struct aggregation** and **struct embedding**.

### Struct Aggregation

This approach implies a `has-a` relationship. For example, a line has two points (start and end) and can be declared as follows:

```go
// Line struct has two fields of type Point
type Line struct {
	Start, End Point
}

// Distance methods calculates the euclidean distance 
// between the two Points
func (l Line) Distance() float64 {
	xDiff := math.Abs(l.Start.X - l.End.X)
	yDiff := math.Abs(l.Start.Y - l.End.Y)

	return math.Sqrt(math.Pow(xDiff, 2) + math.Pow(yDiff, 2))
}

// Usage
func main() {
	l := Line{
		Start: Point{
			X: 2.304,
			Y: 4.504,
		},
		End: Point{
			X: 30.607,
			Y: 44.104,
		},
	}
	fmt.Printf("Distance from %v to %v is %f units\n", l.Start, l.End, l.Distance())
}
// Distance from Point {2.304000, 4.504000} to Point {30.607000, 44.104000} is 48.674632 units
```

Let's redefine `Line` struct slightly differently using inline structs,
```go
type Line struct {
	Start, End struct {
		X float64
		Y float64
	}
}

// Usage
func main() {
	l := Line{
		Start: struct {
			X float64
			Y float64
		}{
			X: 3.123,
			Y: 8.123,
		},
		End: struct {
			X float64
			Y float64
		}{
			X: 4.123,
			Y: 7.123,
		},
	}
}
```
This approach requires you to rely on `anonymous structs` during initialization.

### Struct Embedding

This approach implies an `is-a` relationship. For example, a rectangle is a polygon and can be decalred as shown below:
```go
// embed.go
// Polygon has just two fields for the sake of simplicity
type Polygon struct {
	Width, Height int
}

// Set width and height of the polygon
func (p *Polygon) Set(w, h int) {
	p.Width = w
	p.Height = h
}

// Rectangle is a polygon, with one extra field 'color'
type Rectangle struct {
	color string
	Polygon // Notice the embedding?
}

// Area method can access the fields Width and Height even though
// they are not directly defined within the Rectangle struct
func (r *Rectangle) Area() float64 {
	return float64(r.Width * r.Height)
}

func main() {
	var rect Rectangle
	rect.Set(10, 20) // direct
	rect.color = "Blue"
	fmt.Printf("Rectangle: %+v\n", rect)
	fmt.Printf("Rectangle Width: %+v\n", rect.Width) // direct
	fmt.Printf("Rectangle Height: %+v\n", rect.Height) // direct
	fmt.Printf("Area of the rectangle is: %+v\n", rect.Area())

	rect.Polygon.Set(100, 200) // indirect
	fmt.Printf("Rectangle: %+v\n", rect)
}
``` 
results in,
```sh
$ go run embed.go
Rectangle: {color:Blue Polygon:{Width:10 Height:20}}
Rectangle Width: 10
Rectangle Height: 20
Area of the rectangle is: 200
Rectangle: {color:Blue Polygon:{Width:100 Height:200}}
```

You can see that `rect` can access:
1. `Rectangle` struct's fields and methods
2. `Polygon` struct's fields and methods - both directly and indirectly

I am not sure if that's the correct way to put it, but these are my observations.

---

Note: _This article is not an in-depth tutorial or treatment of Golang's syntax, semantics, design or implementation, but a journal of my learnings._

Struct aggregation and embedding in golang

Up And Running With Kafka On AWS EKS Using Strimzi


import MDImage from '../../../components/MDImage.js';

# Accessing Amazon RDS From AWS EKS

## Contents <a id="contents"></a>

1.  [Premise](#premise)
2.  [Setup the MySQL Database - Amazon RDS](#setup-the-mysql-database)
    1.  [Create the VPC](#rds-create-the-vpc)
    2.  [Create the subnets](#rds-create-subnets)
    3.  [Create the DB subnet group](#rds-create-db-subnet-group)
    4.  [Create the VPC security group](#rds-create-vpc-security-group)
    5.  [Create a DB instance in the VPC](#rds-create-db-instance-in-the-vpc)
    6.  [Amazon RDS setup diagram](#rds-setup-diagram)
3.  [Setup the EKS cluster](#setup-the-eks-cluster)
4.  [Let's build the bridge!](#lets-build-the-bridge)
    1.  [Create and Accept a VPC Peering Connection](#create-and-accept-vpc-peering-connections)
    2.  [Update the EKS cluster VPC's route table](#update-eks-cluster-vpc-route-table)
    3.  [Update the RDS VPC's route table](#update-rds-vpc-route-table)
    4.  [Update the RDS instance's security group](#update-rds-instance-security-group)
5.  [Test the connection](#test-the-connection)

## 1. Premise <a id="premise"></a>

When moving your services to the Kubernetes ecosystem for the first time, it is best practice to port only the stateless parts to begin with.

Here's the problem I had to solve: Our service uses [Amazon RDS for MySQL][1]. Both the RDS instance(s) and EKS reside within their own dedicated [VPC][2]. How do resources running within AWS EKS communicate with the database?

<MDImage src="/0/1_9f54sopkrm2za71fhvl5.png" alt="Problem visualised" />

Let's dive right in!

## 2. Setup the MySQL Database (Amazon RDS) <a id="setup-the-mysql-database"></a>

We will be using the AWS CLI for setting up MySQL database.

### 2.1 Create the VPC <a id="rds-create-the-vpc"></a>

We will first create a VPC with the CIDR block `10.0.0.0/24` which accommodate 254 hosts in all. This is **more than enough** to host our RDS instance.

```sh
$ aws ec2 create-vpc --cidr-block 10.0.0.0/24 | jq '{VpcId:.Vpc.VpcId,CidrBlock:.Vpc.CidrBlock}'
{
    "VpcId": "vpc-0cf40a5f6db5eb3cd",
    "CidrBlock": "10.0.0.0/24"
}

# Export the RDS VPC ID for easy reference in the subsequent commands
$ export RDS_VPC_ID=vpc-0cf40a5f6db5eb3cd
```

### 2.2 Create the subnets <a id="rds-create-subnets"></a>

RDS instances launched in a VPC must have a [DB subnet group][3]. DB subnet groups are a collection of subnets within a VPC. Each DB subnet group should have `subnets` in at least two `Availability Zones` in a given `AWS Region`.

We will divide the RDS VPC (`RDS_VPC_ID`) into two equal subnets: `10.0.0.0/25` and `10.0.0.128/25`.

So, let's create the first subnet in the availability zone `ap-south-1b`:

```sh
$ aws ec2 create-subnet --availability-zone "ap-south-1b" --vpc-id ${RDS_VPC_ID} --cidr-block 10.0.0.0/25 | jq '{SubnetId:.Subnet.SubnetId,AvailabilityZone:.Subnet.AvailabilityZone,CidrBlock:.Subnet.CidrBlock,VpcId:.Subnet.VpcId}'
# Response:
{
  "SubnetId": "subnet-042a4bee8e92287e8",
  "AvailabilityZone": "ap-south-1b",
  "CidrBlock": "10.0.0.0/25",
  "VpcId": "vpc-0cf40a5f6db5eb3cd"
}

```

and the second one in the availability zone `ap-south-1a`

```sh
$ aws ec2 create-subnet --availability-zone "ap-south-1a" --vpc-id ${RDS_VPC_ID} --cidr-block 10.0.0.128/25 | jq '{SubnetId:.Subnet.SubnetId,AvailabilityZone:.Subnet.AvailabilityZone,CidrBlock:.Subnet.CidrBlock,VpcId:.Subnet.VpcId}'
# Response:
{
  "SubnetId": "subnet-0c01a5ba480b930f4",
  "AvailabilityZone": "ap-south-1a",
  "CidrBlock": "10.0.0.128/25",
  "VpcId": "vpc-0cf40a5f6db5eb3cd"
}
```

Each VPC has an implicit router which controls where network traffic is directed. Each subnet in a VPC must be explicitly associated with a route table, which controls the routing for the subnet.

Let's go ahead and associate these two subnet that we created, to the VPC's route table:

```sh
# Fetch the route table information
$ aws ec2 describe-route-tables --filters Name=vpc-id,Values=${RDS_VPC_ID} | jq '.RouteTables[0].RouteTableId'
"rtb-0e680357de97595b1"

# For easy reference
$ export RDS_ROUTE_TABLE_ID=rtb-0e680357de97595b1

# Associate the first subnet with the route table
$ aws ec2 associate-route-table --route-table-id rtb-0e680357de97595b1 --subnet-id subnet-042a4bee8e92287e8
{
    "AssociationId": "rtbassoc-02198db22b2d36c97"
}

# Associate the second subnet with the route table
$ aws ec2 associate-route-table --route-table-id rtb-0e680357de97595b1 --subnet-id subnet-0c01a5ba480b930f4
{
    "AssociationId": "rtbassoc-0e5c3959d360c92ab"
}

```

### 2.3 Create DB Subnet Group <a id="rds-create-db-subnet-group"></a>

Now that we have two subnets spanning two availability zones, we can go ahead and create the **DB subnet group**.

```sh
$ aws rds create-db-subnet-group --db-subnet-group-name  "DemoDBSubnetGroup" --db-subnet-group-description "Demo DB Subnet Group" --subnet-ids "subnet-042a4bee8e92287e8" "subnet-0c01a5ba480b930f4" | jq '{DBSubnetGroupName:.DBSubnetGroup.DBSubnetGroupName,VpcId:.DBSubnetGroup.VpcId,Subnets:.DBSubnetGroup.Subnets[].SubnetIdentifier}'
# Response:
{
  "DBSubnetGroupName": "demodbsubnetgroup",
  "VpcId": "vpc-0cf40a5f6db5eb3cd",
  "Subnets": "subnet-0c01a5ba480b930f4"
}
{
  "DBSubnetGroupName": "demodbsubnetgroup",
  "VpcId": "vpc-0cf40a5f6db5eb3cd",
  "Subnets": "subnet-042a4bee8e92287e8"
}
```

### 2.4 Create a VPC Security Group <a id="rds-create-vpc-security-group"></a>

The _penultimate_ step to creating the DB instance is creating a VPC security group, an instance level virtual firewall with _rules_ to control inbound and outbound traffic.

```
$ aws ec2 create-security-group --group-name DemoRDSSecurityGroup --description "Demo RDS security group" --vpc-id ${RDS_VPC_ID}
{
    "GroupId": "sg-06800acf8d6279971"
}

# Export the RDS VPC Security Group ID for easy reference in the subsequent commands
$ export RDS_VPC_SECURITY_GROUP_ID=sg-06800acf8d6279971
```

We will use this security group at a later point, to set an `inbound` rule to allow all traffic from the EKS cluster to the RDS instance.

### 2.5 Create a DB Instance in the VPC <a id="rds-create-db-instance-in-the-vpc"></a>

```sh
$ aws rds create-db-instance \
  --db-name demordsmyqldb \
  --db-instance-identifier demordsmyqldbinstance \
  --allocated-storage 10 \
  --db-instance-class db.t2.micro \
  --engine mysql \
  --engine-version "5.7.26" \
  --master-username demoappuser \
  --master-user-password demoappuserpassword \
  --no-publicly-accessible \
  --vpc-security-group-ids ${RDS_VPC_SECURITY_GROUP_ID} \
  --db-subnet-group-name "demodbsubnetgroup" \
  --availability-zone ap-south-1b \
  --port 3306 | jq '{DBInstanceIdentifier:.DBInstance.DBInstanceIdentifier,Engine:.DBInstance.Engine,DBName:.DBInstance.DBName,VpcSecurityGroups:.DBInstance.VpcSecurityGroups,EngineVersion:.DBInstance.EngineVersion,PubliclyAccessible:.DBInstance.PubliclyAccessible}'

# Respone:
{
  "DBInstanceIdentifier": "demordsmyqldbinstance",
  "Engine": "mysql",
  "DBName": "demordsmyqldb",
  "VpcSecurityGroups": [
    {
      "VpcSecurityGroupId": "sg-06800acf8d6279971",
      "Status": "active"
    }
  ],
  "EngineVersion": "5.7.26",
  "PubliclyAccessible": false
}
```

We can verify that the DB instance has been created in the UI as well:

<MDImage src="/0/2_5reunqhdinqcp3vlrbq4.png" alt="RDS MySQL DB Instance Details" />


### 2.6 Amazon RDS setup diagram <a id="rds-setup-diagram"></a>

<MDImage src="/0/3_feg2ujod4ja6vtudique.jpeg" alt="AWS RDS Setup Diagram" />


## 3. Setup the EKS cluster <a id="setup-the-eks-cluster"></a>

Spinning up an EKS cluster on AWS is as simple as:

```sh
$ eksctl create cluster --name=demo-eks-cluster --nodes=2 --region=ap-south-1
[ℹ]  using region ap-south-1
[ℹ]  setting availability zones to [ap-south-1a ap-south-1c ap-south-1b]
[ℹ]  subnets for ap-south-1a - public:192.168.0.0/19 private:192.168.96.0/19
[ℹ]  subnets for ap-south-1c - public:192.168.32.0/19 private:192.168.128.0/19
[ℹ]  subnets for ap-south-1b - public:192.168.64.0/19 private:192.168.160.0/19
[ℹ]  nodegroup "ng-ae09882f" will use "ami-09c3eb35bb3be46a4" [AmazonLinux2/1.12]
[ℹ]  creating EKS cluster "demo-eks-cluster" in "ap-south-1" region
[ℹ]  will create 2 separate CloudFormation stacks for cluster itself and the initial nodegroup
[ℹ]  if you encounter any issues, check CloudFormation console or try 'eksctl utils describe-stacks --region=ap-south-1 --name=demo-eks-cluster'
[ℹ]  2 sequential tasks: { create cluster control plane "demo-eks-cluster", create nodegroup "ng-ae09882f" }
[ℹ]  building cluster stack "eksctl-demo-eks-cluster-cluster"
[ℹ]  deploying stack "eksctl-demo-eks-cluster-cluster"
[ℹ]  building nodegroup stack "eksctl-demo-eks-cluster-nodegroup-ng-ae09882f"
[ℹ]  --nodes-min=2 was set automatically for nodegroup ng-ae09882f
[ℹ]  --nodes-max=2 was set automatically for nodegroup ng-ae09882f
[ℹ]  deploying stack "eksctl-demo-eks-cluster-nodegroup-ng-ae09882f"
[✔]  all EKS cluster resource for "demo-eks-cluster" had been created
[✔]  saved kubeconfig as "/Users/Bensooraj/.kube/config"
[ℹ]  adding role "arn:aws:iam::account_number:role/eksctl-demo-eks-cluster-nodegroup-NodeInstanceRole-1631FNZJZTDSK" to auth ConfigMap
[ℹ]  nodegroup "ng-ae09882f" has 0 node(s)
[ℹ]  waiting for at least 2 node(s) to become ready in "ng-ae09882f"
[ℹ]  nodegroup "ng-ae09882f" has 2 node(s)
[ℹ]  node "ip-192-168-30-190.ap-south-1.compute.internal" is ready
[ℹ]  node "ip-192-168-92-207.ap-south-1.compute.internal" is ready
[ℹ]  kubectl command should work with "/Users/Bensooraj/.kube/config", try 'kubectl get nodes'
[✔]  EKS cluster "demo-eks-cluster" in "ap-south-1" region is ready

```

We will create a kubernetes `Service` named `mysql-service` of type `ExternalName` aliasing the RDS endpoint `demordsmyqldbinstance.cimllxgykuy3.ap-south-1.rds.amazonaws.com`.

Run `kubectl apply -f mysql-service.yaml` to create the service.

```yaml
# mysql-service.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    app: mysql-service
  name: mysql-service
spec:
  externalName: demordsmyqldbinstance.cimllxgykuy3.ap-south-1.rds.amazonaws.com
  selector:
    app: mysql-service
  type: ExternalName
status:
  loadBalancer: {}
```

Now, clients running inside the pods within the cluster can connect to the RDS instance using `mysql-service`.

Let's test the connect using a throwaway `busybox` pod:

```sh
$ kubectl run -i --tty --rm debug --image=busybox --restart=Never -- sh
If you don't see a command prompt, try pressing enter.
/ # nc mysql-service 3306
^Cpunt!

```

It is evident that the pod is unable to get through! Let's solve the problem now.

## 4. Let's build the bridge! <a id="lets-build-the-bridge"></a>

We are going to create a [VPC Peering Connection][5] to facilitate communication between the resources in the two VPCs. According to the documentation:

> A **VPC peering connection** is a networking connection between two VPCs that enables you to route traffic between them using private IPv4 addresses or IPv6 addresses. Instances in either VPC can communicate with each other as if they are within the same network. You can create a VPC peering connection between your own VPCs, or with a VPC in another AWS account. The VPCs can be in different regions (also known as an inter-region VPC peering connection).

### 4.1 Create and Accept a VPC Peering Connection <a id="create-and-accept-vpc-peering-connections"></a>

To create a VPC peering connection, navigate to:

1. VPC console: https://console.aws.amazon.com/vpc/
2. Select `Peering Connections` and click on `Create Peering Connection`.
3. Configure the details as follows (select the EKS VPC as the `Requester` and the RDS VPC as the `Accepter`):

   <MDImage src="/0/4_5a2ilp9pzgiih073mlvp.png" alt="Configuration" />
4. Click on `Create Peering Connection`
   <MDImage src="/0/5_9yw3pc84bwo31nihvcvg.jpg" alt="Confirmation page" />
5. Select the `Peering Connection` that we just created. Click on `Actions` => `Accept`. Again, in the confirmation dialog box, click on `Yes, Accept`.
   <MDImage src="/0/6_i3pgejmth4xmhb88fybn.jpg" alt="Yes, Accept" />

Don't forget to export the VPC Peering Connection ID:

```bash
$ export VPC_PEERING_CONNECTION_ID=pcx-0cc408e65493fe197
```

### 4.2 Update the EKS cluster VPC's route table <a id="update-eks-cluster-vpc-route-table"></a>

```sh
# Fetch the route table associated with the 3 public subnets of the VPC created by `eksctl`:
$ aws ec2 describe-route-tables --filters Name="tag:aws:cloudformation:logical-id",Values="PublicRouteTable" | jq '.RouteTables[0].RouteTableId'
"rtb-06103bd0704b3a9ee"

# For easy reference
export EKS_ROUTE_TABLE_ID=rtb-06103bd0704b3a9ee

# Add route: All traffic to (destination) the RDS VPC CIDR block is via the VPC Peering Connection (target)
$ aws ec2 create-route --route-table-id ${EKS_ROUTE_TABLE_ID} --destination-cidr-block 10.0.0.0/24 --vpc-peering-connection-id ${VPC_PEERING_CONNECTION_ID}
{
    "Return": true
}
```

### 4.3 Update the RDS VPC's route table <a id="update-rds-vpc-route-table"></a>

```sh
# Add route: All traffic to (destination) the EKS cluster CIDR block is via the VPC Peering Connection (target)
$ aws ec2 create-route --route-table-id ${RDS_ROUTE_TABLE_ID} --destination-cidr-block 192.168.0.0/16 --vpc-peering-connection-id ${VPC_PEERING_CONNECTION_ID}
{
    "Return": true
}
```

### 4.4 Update the RDS instance's security group <a id="update-rds-instance-security-group"></a>

Allow all ingress traffic from the EKS cluster to the RDS instance on port `3306`:

```sh
$ aws ec2 authorize-security-group-ingress --group-id ${RDS_VPC_SECURITY_GROUP_ID} --protocol tcp --port 3306 --cidr 192.168.0.0/16
```

## 5. Test the connection <a id="test-the-connection"></a>

```sh
$ kubectl run -i --tty --rm debug --image=busybox --restart=Never -- sh
If you don't see a command prompt, try pressing enter.
/ # nc mysql-service 3306
N
5.7.26-logR&=lk`xTH???mj	_5#K)>mysql_native_password
```

We can see that `busybox` can now successfully talk to the RDS instance using the service `mysql-service`.

That said, this is what our final setup looks like:
<MDImage src="/0/7_1ba38e5zu8i36egibtvc.jpeg" alt="Final setup" />

**Note**:
This setup allows all pods in the EKS cluster to access the RDS instance. Depending on your use case, this may or may not be ideal to your architecture. To implement more fine-grained access control, considering setting up a [`NetworkPolicy`][6] resource.

Useful resources:

1. [Visual Subnet Calculator][4]
2. [jq - Command-line JSON processor][7]
3. [AWS CLI Command Reference][8]
4. [AWS VPC Peering][5]

[1]: https://aws.amazon.com/rds/mysql/
[2]: https://docs.aws.amazon.com/vpc/latest/userguide/what-is-amazon-vpc.html
[3]: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_VPC.WorkingWithRDSInstanceinaVPC.html#USER_VPC.Subnets
[4]: http://www.davidc.net/sites/default/subnets/subnets.html
[5]: https://docs.aws.amazon.com/vpc/latest/peering/what-is-vpc-peering.html
[6]: https://kubernetes.io/docs/concepts/services-networking/network-policies/
[7]: https://github.com/stedolan/jq
[8]: https://docs.aws.amazon.com/cli/latest/index.html


Ben Sooraj

Blog

18.09.2023

Function Inlining in Golang

28.05.2021

Go RPCs (Remote Procedure Calls)

01.09.2020

The empty interface and the empty struct in golang

10.08.2020

Sending application logs from EKS to Datadog

02.08.2020

Struct aggregation and embedding in golang

09.10.2019

Up And Running With Kafka On AWS EKS Using Strimzi

03.10.2019

Accessing Amazon RDS From AWS EKS