一个很简单的 grpc 程序, 使用 go-zero 生成的代码, 主要用来代替客户端发送 http 请求, 并且返回 response body/response headers, 仅此而已, 压力测试时发现内存不断的上升!没有停止或者下降, 直到到操作系统 kill, 以下是主要代码:
type Request struct {
}
var httpClientPool = sync.Pool{New: func() interface{}{
return &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
InsecureSkipVerify: true,
},
},
}
}}
var bufPool = sync.Pool{
New: func() interface{} {
return bytes.NewBuffer([]byte{})
},
}
func (l *RequestLogic) Request(in *cache.RequestMsg) (*cache.ResponseMsg, error) {
var result = new(cache.ResponseMsg)
req, err := http.NewRequest(in.Method, string(in.Url), bytes.NewReader(in.Body))
if err != nil {
result.ErrorMsg = err.Error()
return result, errors.Wrap(err, "build request error")
}
for k, v := range in.Headers {
req.Header.Add(k, v)
}
client := httpClientPool.Get().(*http.Client)
defer httpClientPool.Put(client)
client.Timeout = request_timeout
if in.ProxyUrl != "" {
u, _ := url.Parse(in.ProxyUrl)
if u != nil {
client.Transport.(*http.Transport).Proxy = http.ProxyURL(u)
}
}
resp, err := client.Do(req)
if err != nil {
return result, errors.Wrap(err, "request error")
}
defer resp.Body.Close()
result.Url = []byte(resp.Request.URL.String())
result.StatusCode = int32(resp.StatusCode)
var w = bufPool.Get().(*bytes.Buffer)
defer bufPool.Put(w)
if err := gz.Compress(resp.Body, w); err != nil {
return result, errors.Wrap(err, "gz compress error")
}
result.IsCompressor = true
result.Body = w.Bytes()
result.Headers = make(map[string]string)
for k, values := range resp.Header {
for _, v := range values {
result.Headers[k] = v
}
}
return result, nil
}
也通过 pprof 查看了, 都是 bytes 分配导致, 但是貌似不是我写的程序问题?
Showing nodes accounting for 930.40MB, 99.07% of 939.12MB total
Dropped 42 nodes (cum <= 4.70MB)
Showing top 10 nodes out of 38
flat flat% sum% cum cum%
760.34MB 80.96% 80.96% 760.34MB 80.96% google.golang.org/protobuf/proto.MarshalOptions.marshal
151.13MB 16.09% 97.06% 151.13MB 16.09% bytes.makeSlice
12.23MB 1.30% 98.36% 12.23MB 1.30% google.golang.org/grpc/internal/transport.newBufWriter (inline)
6.19MB 0.66% 99.02% 6.19MB 0.66% bufio.NewReaderSize (inline)
0.50MB 0.053% 99.07% 18.92MB 2.01% google.golang.org/grpc/internal/transport.NewServerTransport
0 0% 99.07% 150.62MB 16.04% bytes.(*Buffer).Write
0 0% 99.07% 151.13MB 16.09% bytes.(*Buffer).grow
0 0% 99.07% 150.62MB 16.04% compress/flate.(*Writer).Write
0 0% 99.07% 150.62MB 16.04% compress/flate.(*compressor).deflate
0 0% 99.07% 150.62MB 16.04% compress/flate.(*compressor).write
这咋优化呀?还是直接放弃掉 rpc, 构建一个 http 代理服务器?
这是一个专为移动设备优化的页面(即为了让你能够在 Google 搜索结果里秒开这个页面),如果你希望参与 V2EX 社区的讨论,你可以继续到 V2EX 上打开本讨论主题的完整版本。
V2EX 是创意工作者们的社区,是一个分享自己正在做的有趣事物、交流想法,可以遇见新朋友甚至新机会的地方。
V2EX is a community of developers, designers and creative people.