PHP 高并发日志丢失问题

[cs5117155]

我使用的 thinkphp5.0 框架,发现有时候某一个时刻的日志丢失了,但是数据库更新都是正确的。而且我自已封装的日志函数也存在日志丢了的情况，我是否设置文件锁的方式写日志呢,比如flock($fp, LOCK_EX)，有什么办法可以解决日志不丢失的做法

function log($folder = 'debug', $msg)
    {
        $path = LOG_PATH . $folder;
        if (!is_dir($path)) {
            mkdir($path, 0777, true);
        }
        $filename = $path . '/' . date('Ymd') . '.txt';
        $content = date("Y-m-d H:i:s") . "\r\n" . print_r($msg, 1) . "\r\n \r\n \r\n ";
        if (!is_file($filename)) {
            touch($filename);
            chmod($filename, 0777);
        }
        file_put_contents($filename, $content, FILE_APPEND);
    }

[fenglangjuxu]

@fenglangjuxu #19 然后 还可以 用这个 试试 file_put_contents($filename, $content, FILE_APPEND | LOCK_EX);

[cs5117155]

@akira 阿里云的硬件资源指标监控我没有权限进去看，但我通过宝他提供云监控可视化看,负载平均都是在 10%左右,都很稳定的

[cs5117155]

@fenglangjuxu 如果我用 postman 并发请求 1000 个，其实也可以达到这个效果吧

[cs5117155]

@fenglangjuxu
刚刚我用 ab 测试工具,日志里面只记录时间戳，发现最后一共有 5000 行时间戳,
file_put_contents($filename, $content, FILE_APPEND | LOCK_EX);
file_put_contents($filename, $content, FILE_APPEND);
得到的是相同的结果,并没有发现丢失日志的情况
$ ab -n 5000 -c 1000 https://www.test.com/ [13:51:25]
This is ApacheBench, Version 2.3 <$Revision: 1843412 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking www.test.com (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Completed 5000 requests
Finished 5000 requests


Server Software: nginx
Server Hostname: www.test.com
Server Port: 443
SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES128-GCM-SHA256,2048,128
Server Temp Key: X25519 253 bits
TLS Server Name: www.test.com

Document Path: /
Document Length: 0 bytes

Concurrency Level: 1000
Time taken for tests: 38.410 seconds
Complete requests: 5000
Failed requests: 0
Total transferred: 765000 bytes
HTML transferred: 0 bytes
Requests per second: 130.17 [#/sec] (mean)
Time per request: 7682.085 [ms] (mean)
Time per request: 7.682 [ms] (mean, across all concurrent requests)
Transfer rate: 19.45 [Kbytes/sec] received

Connection Times (ms)
min mean[+/-sd] median max
Connect: 18 3162 4908.2 1305 31784
Processing: 5 109 713.2 10 23984
Waiting: 5 104 707.6 10 23984
Total: 25 3271 4997.5 1406 38358

Percentage of the requests served within a certain time (ms)
50% 1406
66% 2480
75% 3593
80% 4551
90% 8914
95% 14342
98% 20763
99% 26161
100% 38358 (longest request)

[fenglangjuxu]

@cs5117155 那你线上说 丢日志 能在日志那里加个计数么 存 redis 之类的

[fenglangjuxu]

你说的数据库 都是正确的 是指日志数量么?
如果是,确保到达了数据库 肯定会到达这个记录 log 么

[akira]

@cs5117155 硬盘 io 的数据是不可信的。 自己实测一下 是多少

[Aluhao]

@cbasil error_log 写入 +1

[Aluhao]

建议不要产生一条写一条到硬盘，而是把整个程序执行过程的日志写入到一个数组里，等程序执行结束后一起写入到硬盘。

每次往 data 函数面写

class log
{
private $lists = [];
public function data($value, $name = 'debug')
{
$this->lists[$name][] = $value;
}

public function __destruct()
{
if (is_array($this->lists)) {
foreach($this->lists as $name => $value) {
if ($value) {
error_log(print_r($value, true) . PHP_EOL, 3, $name . '.log');
}
}
}
}
}