keithball.net

Some blatherings by Keith Ball

View my projects on GitHub

Blog Posts

  • 04 Feb 2019 » AWS Serverless Experiment

    What?

    Serverless computing is a cloud-computing execution model in which the cloud provider runs the server, and dynamically manages the allocation of machine resources. https://en.wikipedia.org/wiki/Serverless_computing

    I would like to try to build a toy project on a serverless platform to see how it all glues together.

    I will experiment first on AWS, as it is the longest and from everythign I have read the most mature serverless platform. It also has a nice free tier for me to play with.

    Why?

    Serverless promises a lot. Lets see what it feels like.

    How?

    I would like to connect AWS Lambda, API Gateway, Amazon Aurora Serverless and Cognito together to create a example application.

    Setting up a golang lambda function to respond via a api gateway request is very easy. I managed to do everything though the web UI of aws.

    Here is the example hello world example taken from the AWS documentation.

    package main

import (
	"errors"
	"log"

	"github.com/aws/aws-lambda-go/events"
	"github.com/aws/aws-lambda-go/lambda"
)

var (
	// ErrNameNotProvided is thrown when a name is not provided
	ErrNameNotProvided = errors.New("no name was provided in the HTTP body")
)

// Handler is your Lambda function handler
// It uses Amazon API Gateway request/responses provided by the aws-lambda-go/events package,
// However you could use other event sources (S3, Kinesis etc), or JSON-decoded primitive types such as 'string'.
func Handler(request events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {

	// stdout and stderr are sent to AWS CloudWatch Logs
	log.Printf("Processing Lambda request %s\n", request.RequestContext.RequestID)

	// If no name is provided in the HTTP request body, throw an error
	if len(request.Body) < 1 {
		return events.APIGatewayProxyResponse{}, ErrNameNotProvided
	}

	return events.APIGatewayProxyResponse{
		Body:       "Hello " + request.Body,
		StatusCode: 200,
	}, nil
}

func main() {
	lambda.Start(Handler)
}

    Once I had done it through the UI, I wanted to then automate the process of updating the function via the the aws cli.

    Build the function

    GOOS=linux go build -o bin/main github.com/keithballdotnet/aws-serverless/functions

    Zip the function

    zip bin/deployment.zip bin/main

    Update the function

    aws lambda update-function-code --function-name helloWorld --zip-file fileb://./bin/deployment.zip --region eu-central-1
  • 25 Oct 2018 » Keith's Management Guide

    Keith’s Management Guide

    Lean Management

    When asked a question, ask yourself if you really have to answer it. The answer is always not yet, so you don’t answer.

    Scrum Management

    When being asked a question, make an estimate about how long the question will take to answer and add it to a prioritised list of questions in a backlog. Inform the asker that you cant answer right now, as your current sprint is full and that with the current velocity the question will be answered in 6 months, with an error rate of +/- ~80%. Explain that the error rate is so high due to the problem of the changing definition of Story Points with relation to rhetorical questions.

    Waterfall Management

    Prepare a list of questions that you expect to answer over the next 6 months, plan how long those questions will take to answer and roughly when you think those questions will be asked. When asked a question that is not on the list, look at the person asking the question with a confused face, and just give them the expected answer for the currently planned question.

    Extreme Management

    When asked a question, immediately gather all teams members, order pizzas and lock yourself in a room for week. Emerge from the room with the answer for a different question.

    Kanban Management

    When asked a question tell them that you can only work on one question at a time, that you are currently busy thinking about what you want for lunch and that you will get back to them at some point in the future. You might tell them after lunch, but at the time you may have switched to the more pressing topic of what you want for dinner.

    Clean Management

    When asked a question, spend 25 years formulating an answer that is beautifully crafted. The elegance of the answer is so wonderful that you print and frame the answer for all to behold. You upload the answer to github and get hundreds of stars and watchers. The person that asked the question has long since died.

    Evolutionary Management

    Ensure any answer that you give can be given to any other question.

    Agile Management

    When asked a question, you start to give the answer but during the response the asker of the question stops you and asks you a different question, you start to give the answer but during the response the asker of the question stops you and asks you a different question, you start to give the answer but during the response the asker of the question stops you and asks you a different question, you start to give the answer but during the response the asker of the question stops you and asks you a different question, you start to give the answer but during the response the asker of the question stops you and asks you a different question, you start to give the answer but during the response the asker of the question stops you and asks you a different question, you start to give the answer but during the response the asker of the question stops you and asks you a different question, you start to give the answer but during the response the asker of the question stops you and asks you a different question, you start to give the answer but during the response the asker of the question stops you and asks you a different question, you start to give the answer but during the response the asker of the question stops you and asks you a different question, …

    Consultant Management

    You get asked a question. You pay a team of specialist consultants 5000€ to teach you how best to answer the question. They assure you that although they have never heard that particular question before, you are now equipped with the right tools to answer any question with trust, respect, honesty and high velocity. Whatever the question was, you always answer “agile”. If the questioner doesn’t accept that answer, tell them they are “anti-agile” and are not “agile enough” to understand the answer. Advise them to go on a training course (with Lego).

    Freelancer Management

    As a freelancer you get asked a question, you are paid by the hour so you keep repeating “it depends” for as long as possible. When you finally see that the askers patience is running out, give the answer but make it so complicated that the questioner has to ask another question to find out what you meant… and repeat.
  • 28 Oct 2016 » Top tip

    Top tip!

    • Sometimes it is ok to throw money at a problem. Especially when travelling.
  • 19 Oct 2016 » How to add a private docker registry to k8s

    If you are getting problem with ImagePullBackOff and the detailed error:

    Failed to pull image “{MYREPO/myservice}”: Error response from daemon: {“message”:”Get https://{SERVER}:{PORT}/v1/_ping: x509: certificate signed by unknown authority”}

    There are a couple of things you can do. The easy way:

    vi /etc/docker/daemon.json
    {
  "insecure-registries": ["{SERVER}:{PORT}"]
}
    systemctl restart docker

    Or add the certifcate to the trusted list. This example is for ubuntu.

    Add the secret

    kubectl create secret docker-registry dockerkey --docker-username={USER} --docker-password={PASSWORD} --docker-email=r{EMAIL} --docker-server={SERVER}:{PORT}

    On the hosts get the self signed cert (if needed):

    ex +'/BEGIN CERTIFICATE/,/END CERTIFICATE/p' <(echo | openssl s_client -showcerts -connect {SERVER}:{PORT}) -scq > cert.crt

    Add the cert to the list.

    cp dev-reg.crt /usr/local/share/ca-certificates

    Update the ca-certificates.

    List catalog:

    sudo update-ca-certificates

    Check the cert is trusted:

    curl https://{SERVER}:{PORT}
  • 12 Oct 2016 » Top tip

    I have decided to keep a collection of tips. I will compile them and then give these tips to my children before I die…

    Top tip!

    • Spend money on good toilet paper
  • 09 Oct 2016 » How to query a remote docker registry

    If you are having problems with kubernetes reporting ErrImageNotFound when pulling an image, you can eliminate the obvious by checking the registry in the following way.

    Do a docker login:

    docker login -u {USERNAME} -p {PASSWORD} {REGISTRY}

    Get your docker authentication token. Take the value for your registry url:

    cat ~/.docker/config.json

    Do some curl operations.

    List catalog:

    curl -X GET -H "Authorization: Basic {YOURTOKEN}" "https://{REGISTRY}/v2/_catalog"

    List tags on an image:

    curl -X GET -H "Authorization: Basic {YOURTOKEN}" "https://{REGISTRY}/v2/{IMAGENAME}/tags/list"
  • 14 Apr 2016 » Base

    “If human nature were not base, but thoroughly honourable, we should in every debate have no other aim than the discovery of truth” - Arthur Schopenhauer
  • 01 Mar 2016 » Go Gotcha

    Go is lexically scoped…

    What is wrong with the following code?

    // Item
var item *Item

itemID := u.Query().Get("itemID")

if itemID != "" {
item, err := repository.GetItem(itemID)
}

log.Printf("The item %v", item)

    The code shows how Item is being redeclared in the scope of the if.

    Further reading here: Declarations and scope
  • 28 Feb 2016 » Meditations

    “Be like a rocky promontory against which the restless surf continually pounds; it stands fast while the churning sea is lulled to sleep at its feet. I hear you say, “How unlucky that this should happen to me!” Not at all! Say instead, “How lucky that I am not broken by what has happened and am not afraid of what is about to happen. The same blow might have struck anyone, but not many would have absorbed it without capitulation or complaint.” - Marcus Aurelius
  • 28 Feb 2016 » Dream Job

    I like this…

    Image of Dream Job
  • 27 Feb 2016 » Go-KMS

    What is GO-KMS?

    GO-KMS is a encryption Key Management Service in GO. Modelled extensively on AWS KMS behaviour, the API is used for symmetrical key management. It offers Cryptography as a Service (CaaS) functionality such as encryption/decryption/reencryption without exposing keys.

    The crypto provider is based on AES and a key size of 256bits using the GCM cipher to provide confidentiality as well as authentication.

    Keys are encrypted and stored on disk, using a master key which is derived using PBKDF2 from a passphrase when run in pure software mode. It is also possible to combine GO-KMS with a Hardware Security Module (HSM) which can be leveraged to create and encrypt a master key using the HSM for generation and protection. HSM support is done using the PKCS#11 standard.

    GO-KMS authentication is done using HMAC-SHA256 over HTTPS.

    // AesGCMEncrypt Encrypt data using AES with the GCM cipher mode (Gives Confidentiality and Authenticity)
func AesGCMEncrypt(plaintext []byte, key []byte) ([]byte, error) {
	block, err := aes.NewCipher(key)
	if err != nil {
		return nil, err
	}

	gcm, err := cipher.NewGCM(block)
	if err != nil {
		return nil, err
	}

	nonce := make([]byte, gcm.NonceSize())
	if _, err := rand.Read(nonce); err != nil {
		return nil, err
	}

	ciphertext := gcm.Seal(nil, nonce, plaintext, nil)

	return append(nonce, ciphertext...), nil
}

    Checkout go-kms on github for more info.