HTTP server from TCP socket (in Go)

Question

I'm trying to create a TCP socket in Go, bind it into a VRF interface and to establish a HTTP server in that specific interface. The VRF binding works correctly, but starting the HTTP server returns an error stating "accept tcp 127.0.0.1:80: accept: invalid argument". Am I right to assume, that the socket is somehow defective and I'm creating it wrong?

Below is the simplified version reproducing the problem. VRF part is commented out as it doesn't affect the actual problem, but I'm leaving it here as I'm trying to avoid people telling me to just use net.Listen instead of sockets. VRF needs to be binded into first before it can be used so net.Listen isn't unfortunately an option.

package main

import (
    "fmt"
    "net"
    "net/http"
    "os"
    "syscall"
)

func main() {
    fd, err := syscall.Socket(syscall.AF_INET, syscall.SOCK_STREAM, syscall.IPPROTO_TCP)
    if err != nil {
        fmt.Printf("Error creating socket: %v", err)
        os.Exit(1)
    }

    // if err = syscall.SetsockoptString(fd, syscall.SOL_SOCKET, syscall.SO_BINDTODEVICE, "vrfiface"); err != nil {
    //  fmt.Printf("Error binding to vrf: %v", err)
    //  os.Exit(1)
    // }

    sockAddr := &syscall.SockaddrInet4{
        Port: 80,
        Addr: [4]byte{127, 0, 0, 1},
    }

    if err = syscall.Bind(fd, sockAddr); err != nil {
        fmt.Printf("Error binding to IP and port: %v", err)
        os.Exit(1)
    }

    file := os.NewFile(uintptr(fd), "socketfile")
    if file == nil {
        fmt.Println("Error creating file")
        os.Exit(1)
    }

    listener, err := net.FileListener(file)
    if err != nil {
        fmt.Printf("Error creating a listener: %v", err)
        os.Exit(1)
    }

    http.HandleFunc("/", TestServer)
    if err = http.Serve(listener, nil); err != nil {
        fmt.Printf("Error serving HTTP requests: %v", err)
    }
}

func TestServer(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Test, %s!", r.URL.Path[1:])
}

Any pointers on solving this would be appreciated. Thank you!

What is wrong with existing web frameworks which could get you started in a minute? — Maxim Sagaydachny
– Maxim Sagaydachny, Commented Feb 21, 2020 at 13:30
Are you certain you need to bind to a specific interface? In Linux you rarely do that because the interface is chosen during routing. You should probably also just use net.ListenConfig to control the connection setup — Mr_Pink
– Mr_Pink, Commented Feb 21, 2020 at 14:44
@JimB: The point of VRF is to have different routing tables per VRF device. One might even have the same IP address on different virtual devices on the same machine and the point is to explicitly bind to the interface which should be used and not let the OS pick some arbitrary one. — Steffen Ullrich
– Steffen Ullrich, Commented Feb 21, 2020 at 18:08
Thanks @SteffenUllrich. I assume then ListenConfig is probably the way to go to avoid the rest of the socket setup once you bind the device — Mr_Pink
– Mr_Pink, Commented Feb 21, 2020 at 18:46

Mr_Pink · Accepted Answer · 2020-02-21 20:08:33Z

5

You can use a net.ListenConfig to inject the socket options you want before syscall.Bind is called. This also makes sure the socket setup is completed correctly, and in the same manner as expected by the net package.

The ListenConfig.Control function gives you a syscall.RawConn on which to call Control with a closure, where you have access to the raw file descriptor being using during the socket setup.

func main() {
    lc := net.ListenConfig{Control: controlOnConnSetup}

    ln, err := lc.Listen(context.Background(), "tcp", "127.0.0.1:80")
    if err != nil {
        log.Fatal(err)
    }
    ln.Close()
}

func controlOnConnSetup(network string, address string, c syscall.RawConn) error {
    var operr error
    fn := func(fd uintptr) {
        operr = syscall.SetsockoptString(int(fd), syscall.SOL_SOCKET, syscall.SO_BINDTODEVICE, "vrfiface")
    }
    if err := c.Control(fn); err != nil {
        return err
    }
    if operr != nil {
        return operr
    }
    return nil
}

edited Feb 21, 2020 at 20:08

answered Feb 21, 2020 at 19:43

Mr_Pink

110k17 gold badges287 silver badges275 bronze badges

Sign up to request clarification or add additional context in comments.

5 Comments

Steffen Ullrich Over a year ago

I've tried something like this too but got "cannot use fd (type uintptr) as type int in argument to syscall.SetsockoptString". I also get this error with your code, i.e. does not compile for me.

Mr_Pink Over a year ago

@SteffenUllrich: sorry, forgot the type conversion in there, I was testing on macOS, so using a different syscall. Should run now.

Steffen Ullrich Over a year ago

Thanks, and actually obvious in hindsight to just cast it to an int.

AjMyyra Over a year ago

@JimB: Btw, is there a way to pass context to the control function? I'd love to pass the VRF interface name from there with context.WithValue or similarly.

Mr_Pink Over a year ago

@AjMyyra, there’s no context here, but you can generate a closure around whatever value need.

Steffen Ullrich · Accepted Answer · 2020-02-21 18:15:57Z

1

As mentioned in a comment by C Han already: you need to listen. The sequence of creating a server is to create the socket, bind to the IP and port, call listen and then accept incoming connections. If you forget to listen then you'll get the error you see. Thus:

if err = syscall.Bind(fd, sockAddr); err != nil {
     ...
}

if err = syscall.Listen(fd, 10); err != nil {
    fmt.Printf("Error listening %v", err)
    os.Exit(1)
}

answered Feb 21, 2020 at 18:15

Steffen Ullrich

125k11 gold badges157 silver badges194 bronze badges

3 Comments

AjMyyra Over a year ago

Thank you! My mistake was assuming that the listener would take care of this for me. Working perfectly now!

Steffen Ullrich Over a year ago

@AjMyyra: I nevertheless recommend that you use the approach from JimB instead since it simplifies the task and lets you use all the comfort of Go while still being able to bind to the device.

AjMyyra Over a year ago

I did, actually. I marked your response as it was the clearest solution to the original problem, but JimB's approach is definitely a better way to handle this.

Collectives™ on Stack Overflow

HTTP server from TCP socket (in Go)

2 Answers 2

5 Comments

3 Comments

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

2 Answers 2

5 Comments

3 Comments

Your Answer

Sign up or log in

Post as a guest

Linked

Related