go 杂项
go 杂项
2023/04/15
http.Response.Body
注意 Body 字段的注释:
The default HTTP client’s Transport may not reuse HTTP/1.x “keep-alive” TCP connections if the Body is not read to completion and closed.
为了连接复用，需要读取和关闭 body，如果实际不用 body 时可使用 io.Copy(io.Discard, resp.Body)
使用 strace 查看连接情况:
strace -qqfe connect ./read_body 2>&1 | grep -E '\(80\)|doRequest'
In [1]:
%%writefile /tmp/read_body.go
// go build -o read_body main.go
package main
import (
"fmt"
"io"
"net/http"
)
func doRequest(closeBody, readBody bool) {
resp, err := http.Get(`http://httpbin.org/get`)
if err != nil {
panic(err)
}
defer resp.Body.Close()
if readBody {
io.Copy(io.Discard, resp.Body)
}
fmt.Printf("doRequest done, closeBody=%v, readBody=%v\n", closeBody, readBody)
}
func main() {
doRequest(true, true)
doRequest(false, true)
doRequest(true, true)
doRequest(true, true)
doRequest(true, false)
doRequest(true, true)
}Out[1]:
Cell contents written to "/tmp/read_body.go".In [2]:
!cd /tmp; go build -o read_body read_body.go
!cd /tmp; strace -qqfe connect ./read_body 2>&1 | grep -E '\(80\)|doRequest'
// 可以看到有两次 `connect(5, {sa_family=AF_INET...`, 第二发生在 `doRequest(true, false)` 的下一个请求, 即其没有释放连接
Out[2]:
[pid 33511] connect(5, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("198.18.1.217")}, 16) = -1 EINPROGRESS (Operation now in progress)
doRequest done, closeBody=true, readBody=true
doRequest done, closeBody=false, readBody=true
doRequest done, closeBody=true, readBody=true
doRequest done, closeBody=true, readBody=true
doRequest done, closeBody=true, readBody=false
[pid 33511] connect(5, {sa_family=AF_INET, sin_port=htons(80), sin_addr=inet_addr("198.18.1.217")}, 16) = -1 EINPROGRESS (Operation now in progress)
doRequest done, closeBody=true, readBody=truejson.Marshal
结构体内部匿名字段实现了 json.Marshaler(方法: MarshalJSON() ([]byte, error)), 则会直接调此 MarshalJSON 而忽略结构体.
源码:
- 创建序列化 encoderFunc: f = newTypeEncoder(t, true)
- 判断是否实现了
json.Marshalerif t.Implements(marshalerType) { return marshalerEncoder }
所以结构体内匿名字段要注意, 如下面 FileWrapper 的序列化会被 File 顶替.
In [3]:
type File struct {
Name string
}
var _ json.Marshaler = (*File) (nil)
func (file *File) MarshalJSON() ([]byte, error) {
b := make([]byte, len(file.Name)+2)
copy(b[1:], []byte(file.Name))
b[0], b[len(b)-1] = '"', '"'
return b, nil
}
type FileWrapper struct {
*File
Dir string
}
func main() {
file := &File{"test.go"}
wrapper := &FileWrapper{file, "workspace"}
bytes1, _ := json.Marshal(file)
fmt.Println("file json:", string(bytes1))
bytes2, _ := json.Marshal(wrapper)
fmt.Println("wrapper json:", string(bytes2))
}Out[3]:
file json: "test.go"
wrapper json: "test.go"json.Unmarshal 解析数值到 interface
数值类型序列化到 interface, 会默认当作 float64 来解析, 源码:
In [4]:
%%
raw := `{"key": 1}`
var data map[string]interface{}
_ = json.Unmarshal([]byte(raw), &data)
// 可以看到类型是 float64
fmt.Printf("%T, %v\n", data["key"], data["key"])
// 数字延迟解析, 先存原始字符串到 Number
decoder := json.NewDecoder(strings.NewReader(raw))
decoder.UseNumber()
decoder.Decode(&data)
// 可以看到类型是 Number
fmt.Printf("%T, %v\n", data["key"], data["key"])
vint64, _ := data["key"].(json.Number).Int64()
fmt.Printf("to int64: %v\n", vint64)Out[4]:
float64, 1
json.Number, 1
to int64: 1所以遇到长整型反序列化成 interface 时要注意精度. 小数部分超过 53 位则有可能丢失精度.
IEEE 754 64-bit floating-point numbers
format.svg 公式是: $(-1)^{\text {sign }}\left(1 . b_{51} b_{50} \ldots b_{0}\right)_{2} \times 2^{e-1023}$, 即先将数字转成二进制,
(-1)^sign * 1.b_xx * 2^E.十进制小数表示流程: 2.25 转二进制过程: 整数部分是 2, 通过"除2取余数"得到二进制
10, 小数部分是 0.25, 通过"乘2取整数"得到二进制01, 所以 $2.25$ 的二进制是 $10.01$, 科学计数法表示为 $1.001 × 2^1$,
- 符号位: 是正数所以是 0
- 指数部分: $1.001 × 2^1$ 得到指数是 1, 则存储的指数是 $1 + 1023 = 1024$, 二进制是
10000000000- 小数部分: $1.001 × 2^1$ 得到小数是
001, 则存储的小数是001右侧补 0 直到长度为 52
In [5]:
// 查看 float64 的二进制表示
func showfloat64bits(f float64) {
bits := fmt.Sprintf("%064v", strconv.FormatUint(math.Float64bits(f), 2))
fmt.Printf("float64(%v)\n\tbits: %064v\n", f, bits)
fmt.Printf("\tsign=%v, exponent=%v, fraction=%v\n", string(bits[0]), bits[1:12], bits[12:])
}
%%
showfloat64bits(2.25)Out[5]:
float64(2.25)
bits: 0100000000000010000000000000000000000000000000000000000000000000
sign=0, exponent=10000000000, fraction=0010000000000000000000000000000000000000000000000000可以看到 2 ^ 53 + i (二进制表示为 $1.\overbrace{0\cdot\cdot\cdot \text{X}1 }^{53\text{位}}$) 当 i=1,3,5,… 时会丢失精度, 原因是这些数的小数部分二进制表示是 0...X[1] 共计 53 位取有效的前 52 位, 而被丢弃的最后一位是 1 不是 0.
In [6]:
%%
start := 1 << 53
for i := range 4 {
vint := start + i
fmt.Printf("\nint= 2 ^ 53 + %v\n", i)
vfloat64 := float64(vint)
vint_n := int(vfloat64)
if vint != vint_n {
fmt.Printf("有丢失: %v\n", vint_n)
} else {
fmt.Printf("无丢失\n")
}
showfloat64bits(vfloat64)
}Out[6]:
int= 2 ^ 53 + 0
无丢失
float64(9.007199254740992e+15)
bits: 0100001101000000000000000000000000000000000000000000000000000000
sign=0, exponent=10000110100, fraction=0000000000000000000000000000000000000000000000000000
int= 2 ^ 53 + 1
有丢失: 9007199254740992
float64(9.007199254740992e+15)
bits: 0100001101000000000000000000000000000000000000000000000000000000
sign=0, exponent=10000110100, fraction=0000000000000000000000000000000000000000000000000000
int= 2 ^ 53 + 2
无丢失
float64(9.007199254740994e+15)
bits: 0100001101000000000000000000000000000000000000000000000000000001
sign=0, exponent=10000110100, fraction=0000000000000000000000000000000000000000000000000001
int= 2 ^ 53 + 3
有丢失: 9007199254740996
float64(9.007199254740996e+15)
bits: 0100001101000000000000000000000000000000000000000000000000000010
sign=0, exponent=10000110100, fraction=0000000000000000000000000000000000000000000000000010Last updated on