请教一个 go tool ppof 的 heap inuse_space 和 Linux 系统 top 命令显示的内存占用差距过大的问题

2019-11-05 19:06:10 +08:00
 piapia123

现在一个 go 程序存在内存泄露的问题,于是用 go tool ppof 查看了 heap 的 inuse_space 信息

Build ID: 22891c5544961cf88d62b13741ef587dc95d0dcb
Type: inuse_space
Time: Nov 4, 2019 at 8:31pm (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 1.78GB, 96.47% of 1.85GB total
Dropped 84 nodes (cum <= 0.01GB)
Showing top 10 nodes out of 47
      flat  flat%   sum%        cum   cum%
    1.59GB 85.87% 85.87%     1.59GB 85.87%  github.com/derekparker/trie.(*Node).NewChild
    0.03GB  1.64% 87.51%     0.04GB  1.91%  gopkg.in/olivere/elastic%2ev5.(*BulkIndexRequest).Source
    0.03GB  1.43% 88.94%     0.03GB  1.43%  github.com/segmentio/kafka-go.readNewBytes
    0.03GB  1.40% 90.34%     0.03GB  1.40%  bufio.(*Scanner).Text
    0.02GB  1.35% 91.69%     0.02GB  1.35%  main.reverse
    0.02GB  1.19% 92.88%     0.02GB  1.19%  github.com/segmentio/kafka-go.newWriter
    0.02GB  1.11% 93.99%     0.02GB  1.11%  bytes.(*Buffer).String
    0.02GB  0.91% 94.90%     0.02GB  0.94%  github.com/segmentio/kafka-go/snappy.(*xerialReader).readChunk
    0.02GB  0.87% 95.78%     1.65GB 89.49%  main.(*A).Load
    0.01GB  0.69% 96.47%     0.01GB  0.69%  bytes.makeSlice
(pprof)

也看了相应的 alloc_space 信息

Build ID: 22891c5544961cf88d62b13741ef587dc95d0dcb
Type: alloc_space
Time: Nov 4, 2019 at 8:31pm (CST)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 77.60GB, 73.03% of 106.27GB total
Dropped 259 nodes (cum <= 0.53GB)
Showing top 10 nodes out of 90
      flat  flat%   sum%        cum   cum%
   23.02GB 21.66% 21.66%    23.02GB 21.66%  bytes.makeSlice
    9.13GB  8.59% 30.25%     9.13GB  8.59%  github.com/segmentio/kafka-go.readNewBytes
    7.42GB  6.99% 37.23%    24.70GB 23.24%  gopkg.in/olivere/elastic%2ev5.(*BulkIndexRequest).Source
    6.15GB  5.79% 43.02%     6.15GB  5.79%  github.com/gogo/protobuf/proto.unmarshalBytesValue
    6.08GB  5.72% 48.75%     6.30GB  5.93%  github.com/mailru/easyjson/buffer.(*Buffer).BuildBytes
    6.05GB  5.69% 54.44%    17.28GB 16.26%  encoding/json.Marshal
    5.81GB  5.46% 59.90%     5.81GB  5.46%  bytes.(*Buffer).String
    5.74GB  5.40% 65.30%    11.47GB 10.80%  gopkg.in/olivere/elastic%2ev5.(*BulkService).bodyAsString
    4.54GB  4.27% 69.57%     4.54GB  4.27%  github.com/gogo/protobuf/proto.unmarshalStringPtr
    3.67GB  3.45% 73.03%     3.67GB  3.45%  reflect.New
(pprof)

可以看到 inuse_space 一共 1.85G 左右,而实际用 top 看进程使用的内存一直是在缓慢增加的(系统一共 64G 内存,最后可以吃满), 我想知道怎么去看这些增加的内存被用到哪去了,应该怎么排查这个问题。

1757 次点击
所在节点    程序员
2 条回复
reus
2019-11-05 21:52:27 +08:00
既然 go 堆没有增长,那肯定是 C 堆了,看看 cgo 哪里忘记释放内存了。

能不用 cgo,就没必要用 cgo,编译慢,难调试难优化。
piapia123
2019-11-06 10:35:55 +08:00
@reus 多谢大佬,的确是用了 cgo

这是一个专为移动设备优化的页面(即为了让你能够在 Google 搜索结果里秒开这个页面),如果你希望参与 V2EX 社区的讨论,你可以继续到 V2EX 上打开本讨论主题的完整版本。

https://www.v2ex.com/t/616609

V2EX 是创意工作者们的社区,是一个分享自己正在做的有趣事物、交流想法,可以遇见新朋友甚至新机会的地方。

V2EX is a community of developers, designers and creative people.

© 2021 V2EX