Go_Gin檔案上傳 & 資料綁定和驗證

網頁或者是業務上總是會需要讓客戶上傳點檔案的.
像是大頭照、履歷檔:)、謎片:)、帳單PDF

以前Node我都是用Multer在處理這部份.
這次來寫看看Gin的檔案上傳的部份, 會有單檔和多檔案.
玩看看.

Multipart/form-data

提到檔案上傳一定要稍微認識一下這個content-type.
目的用來提高binary檔案的傳輸效率用.
該機制在1998年的RFC2388中被定義出來.

--method POST \
--header 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
--body-data '------WebKitFormBoundary7MA4YWxkTrZu0gW\r\nContent-Disposition: form-data; name="avatar"; filename="Webp.net-resizeimage.png"\r\nContent-Type: image/png\r\n\r\n\r\n------WebKitFormBoundary7MA4YWxkTrZu0gW

Header內多了一串boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW
----WebKitFormBoundary7MA4YWxkTrZu0gW這叫分隔符,
分隔多個文件或者是單檔案的屬性.
接著body中針對該檔案的部份, 每一行開頭都會是分隔符作開頭
下一行緊接著才是要描述該檔案的metadata, form的名稱,檔名,檔案類型…

檔案上傳

multipart/FileHeader

Go裡面用FileHeader這結構體來表示上傳上來的檔案.
然後有個Open()被呼叫後會返回File這組interface, 裡面組合了4組interface.
就會有各種讀取查找跟關檔的實作了.

// A FileHeader describes a file part of a multipart request.
type FileHeader struct {
	Filename string
	Header   textproto.MIMEHeader
	Size     int64

	content []byte
	tmpfile string
}
// Open opens and returns the FileHeader's associated File.
func (fh *FileHeader) Open() (File, error) {...}

// File is an interface to access the file part of a multipart message.
// Its contents may be either stored in memory or on disk.
// If stored on disk, the File's underlying concrete type will be an *os.File.
type File interface {
	io.Reader
	io.ReaderAt
	io.Seeker
	io.Closer
}

先用Gin收下我們上傳的檔案.
新增一個fileHandlder.go, SetupRouter記得註冊路由.
上傳什麼就回應該檔案的metadata.

package handler

import (
	"net/http"

	"github.com/gin-gonic/gin"
)

func UploadSingleIndex(ctx *gin.Context) {
	file, err := ctx.FormFile("file")
	if err != nil {
		ctx.JSON(http.StatusInternalServerError, gin.H{
			"error": err,
		})
	}
	ctx.JSON(http.StatusOK, gin.H{
		"fileName": file.Filename,
		"size":     file.Size,
		"mimeType": file.Header,
	})
}

好, 有成功吃到檔案的描述了.
接著來寫檔吧, 再做一隻API給外部讀取.
總是會需要上傳大頭照, 然後顯示在網頁上的吧.
或者是後台上傳電子帳單, 加上浮水印給用戶下載.

開一個資料夾叫做file, 在專案根目錄.
改寫handler.go.
透過SaveUploadedFile(), 一開始就會呼叫file.Open(), 這會返回上面定義的File interface{}, 這裡返回的是io.SectionReader這類型.
然後呼叫有實做Close()接口的對象, 也就是呼叫SectionReader的Close().

SaveUploadedFile()的第一個參數要是FileHeaer.
第二個則是目標路徑, 這路徑是相對路徑+檔名.

// SaveUploadedFile uploads the form file to specific dst.
func (c *Context) SaveUploadedFile(file *multipart.FileHeader, dst string) error {
	src, err := file.Open()
    ...
	defer src.Close()
    ...
}

func UploadSingleIndex(ctx *gin.Context) {
	file, err := ctx.FormFile("file")
	if err != nil {
		ctx.JSON(http.StatusInternalServerError, gin.H{
			"error": err,
		})
	}

	err = ctx.SaveUploadedFile(file, "./file/"+"demo.png")
	if err != nil {
		ctx.JSON(http.StatusInternalServerError, gin.H{
			"error": err,
		})
	}
	ctx.JSON(http.StatusOK, gin.H{
		"fileName": file.Filename,
		"size":     file.Size,
		"mimeType": file.Header,
	})
}

寫檔成功XD

但企業通常不會這樣存在本機.
都會存在外部的file server或者是AWS的S3.
且過程中可能還會壓縮等等的過程.

來寫測試!!
在test資料夾下, 開個img和file資料夾.
把要上傳的圖片放在img內.
fileRouter_test.go

package test

import (
	"bytes"
	"io"
	"mime/multipart"
	"net/http"
	"net/http/httptest"
	"os"
	"path/filepath"
	"testing"

	"github.com/gin-gonic/gin"
	"github.com/stretchr/testify/assert"
	"github.com/tedmax100/gin-angular/router"
)


func TestUploadSingleRouter(t *testing.T) {
	gin.SetMode(gin.TestMode)
	engine := router.SetupRouter()

    // 開檔
	file, err := os.Open("./img/test.png")
	if err != nil {
		t.Error(err)
	}
	defer file.Close()

	body := &bytes.Buffer{}
    // 產生boundary
	writer := multipart.NewWriter(body)

    // 讀檔並寫到body, 填寫form的field key跟一些內容
	part, err := writer.CreateFormFile("file", filepath.Base("./img/test.png"))
	if err != nil {
		t.Error(err)
	}
	_, err = io.Copy(part, file)
	if err != nil {
		t.Error(err)
	}
	_ = writer.Close()

	w := httptest.NewRecorder()
	req, _ := http.NewRequest(http.MethodPost, "/file/uploadSingle", body)
	req.Header.Add("Content-type", writer.FormDataContentType())

	engine.ServeHTTP(w, req)

	assert.Equal(t, http.StatusOK, w.Code)
}

appleboy大大有寫一套更簡便的API測試工具,也支援檔案上傳
gofight

來用gofight把上面的改寫

package test

import (
	"net/http"
	"testing"

	"github.com/appleboy/gofight/v2"
	"github.com/gin-gonic/gin"
	"github.com/stretchr/testify/assert"
	"github.com/tedmax100/gin-angular/router"
)

func TestUploadSingleRouter(t *testing.T) {
	gin.SetMode(gin.TestMode)
	engine := router.SetupRouter()

	r := gofight.New()
	r.POST("/file/uploadSingle").SetDebug(true).SetFileFromPath([]gofight.UploadFile{
		{
			Path: "./img/test.png",
			Name: "file",
		},
	}).Run(engine, func(r gofight.HTTPResponse, rq gofight.HTTPRequest) {
		assert.Equal(t, http.StatusOK, r.Code)
	})
}

清爽多了QQ
其實Go也能串式調用, 就是只要返回值都是一樣的類型就能了.
但當然要沒有error, 不然會出現side effect.

資料與模型的綁定

我們在操作API來的表單資料時, 是也能自己慢慢取值出來驗證.
但要是可以直接轉成對應的struct. 在操作上就方便許多了.
Gin有提供這樣的機制.

新增一個model資料夾, 裡面新增userLogin.go
透過之前在MySQL那邊提到的tag, 這裡用form這個tag說明在form裡面對應的Key.

package model

type UserLogin struct {
	Email         string `form:"email"`
	Password      string `form:"password"`
	PasswordAgain string `form:"password-again"`
}

修改userHandler.go, 新增UserLogin().
這裡呼叫ShouldBind(), 傳入對應物件的指針.
還有個很像的API叫做Bind, 差別在Bind只要error, 直接就是回傳400.
這兩個API都是透過content-type在做判別再回傳對應的Binding接口來操作.

func (c *Context) ShouldBind(obj interface{}) error {
	b := binding.Default(c.Request.Method, c.ContentType())
	return c.ShouldBindWith(obj, b)
}

func (c *Context) Bind(obj interface{}) error {
	b := binding.Default(c.Request.Method, c.ContentType())
	return c.MustBindWith(obj, b)
}


func Default(method, contentType string) Binding {
	if method == "GET" {
		return Form
	}

	switch contentType {
	case MIMEJSON:
		return JSON
	case MIMEXML, MIMEXML2:
		return XML
	case MIMEPROTOBUF:
		return ProtoBuf
	case MIMEMSGPACK, MIMEMSGPACK2:
		return MsgPack
	case MIMEYAML:
		return YAML
	case MIMEMultipartPOSTForm:
		return FormMultipart
	default: // case MIMEPOSTForm:
		return Form
	}
}
const (
	MIMEJSON              = "application/json"
	MIMEHTML              = "text/html"
	MIMEXML               = "application/xml"
	MIMEXML2              = "text/xml"
	MIMEPlain             = "text/plain"
	MIMEPOSTForm          = "application/x-www-form-urlencoded"
	MIMEMultipartPOSTForm = "multipart/form-data"
	MIMEPROTOBUF          = "application/x-protobuf"
	MIMEMSGPACK           = "application/x-msgpack"
	MIMEMSGPACK2          = "application/msgpack"
	MIMEYAML              = "application/x-yaml"
)

func UserLogin(ctx *gin.Context) {
	var user model.UserLogin
	if err := ctx.ShouldBind(&user); err != nil {
		ctx.JSON(http.StatusBadRequest, err)
		return
	}

	ctx.JSON(http.StatusOK, user)
}

注意, 如果model有屬性是小寫開頭, 就算名稱跟form的key一致, 也沒法綁定上去.

資料驗證

表單傳來的資料是否如我們所要的, 我們也是能逐項存取來驗證.
但Gin透過第三方套件Validator作這部份的驗證.

validators ang tags

剛剛的UserLogin model, 也許我們要自己比較, 但這裡修改一下套用validator.
因為想要Password跟PasswordAgain, 必須要一樣.
這裡直接使用Cross-Field Validation中的eqfield(equal other field).
指名要跟哪個field比較.

type UserLogin struct {
	Email         string `form:"email" binding:"email"`
	Password      string `form:"password" binding:"required"`
	PasswordAgain string `form:"password-again" binding:"eqfield=Password"`
}

再執行看看, 直接回400 跟錯誤訊息

綁定跟驗證都是透過這隻Bind()
驗證結構體則是透過ValidateStruct()
gin/binding/form.go

func (formMultipartBinding) Bind(req *http.Request, obj interface{}) error {
	if err := req.ParseMultipartForm(defaultMemory); err != nil {
		return err
	}
	if err := mappingByPtr(obj, (*multipartRequest)(req), "form"); err != nil {
		return err
	}

	return validate(obj)
}

gin/binding/default_validator.go

// ValidateStruct receives any kind of type, but only performed struct or pointer to struct type.
func (v *defaultValidator) ValidateStruct(obj interface{}) error {
	value := reflect.ValueOf(obj)
	valueType := value.Kind()
	if valueType == reflect.Ptr {
		valueType = value.Elem().Kind()
	}
	if valueType == reflect.Struct {
		v.lazyinit()
		if err := v.validate.Struct(obj); err != nil {
			return err
		}
	}
	return nil
}

加入多個validation, 用,依序隔開

package model

type UserLogin struct {
	Email         string `form:"email" binding:"email"`
	Password      string `form:"password" binding:"required"`
	PasswordAgain string `form:"password-again" binding:"required,eqfield=Password"`
}

補個測試

func TestIUserLoginRouter(t *testing.T) {
	value := url.Values{}
	value.Add("email", "ithome@ithome.com")
	value.Add("password", "ironman")
	value.Add("password-again", "ironman")

	router := router.SetupRouter()

	w := httptest.NewRecorder()
	req, _ := http.NewRequest(http.MethodPost, "/user/login", bytes.NewBufferString(value.Encode()))
	req.Header.Add("Content-Type", "application/x-www-form-urlencoded")
	router.ServeHTTP(w, req)

	assert.Equal(t, http.StatusOK, w.Code)
}

鐵人賽連結