shiznet 最近的时间轴更新 shiznet 最近的时间轴更新 |
shiznetV2EX 第 53731 号会员，加入于 2014-01-08 08:58:06 +08:00 |
| 如何实现一个基于 pattern 的文本相似度聚类 问与答 • shiznet • 2018-06-08 13:48:12 PM • 最后回复来自 takato | 12 |
| MacBookAir 特定无线网络下不稳定 MacBook Air • shiznet • 2015-10-02 15:34:29 PM • 最后回复来自 thank243 | 30 |
| Tracing 系统 or 人肉 Tracing 问与答 • shiznet • 2015-06-25 16:26:26 PM • 最后回复来自 OneAPM | 5 |
shiznet 最近回复了
2018-06-07 12:44:48 +08:00 回复了 GGGG430 创建的主题 › Linux › tail 实时打印日志问题 |
原始需求是什么？
2018-06-06 09:13:20 +08:00 回复了 shiznet 创建的主题 › 问与答 › 如何实现一个基于 pattern 的文本相似度聚类 |
2018-06-06 09:10:36 +08:00 回复了 shiznet 创建的主题 › 问与答 › 如何实现一个基于 pattern 的文本相似度聚类 |
@ETiV
模块 /文件是独立的，这个可以区分开，但是一个模块中可能会输出不同的日志，比如说方法 A 有多个地方会有异常栈输出，且每个异常栈的信息可能略有不同。行数信息是在日志的描述中的一个变量，所以没办法将行数作为直接标识。
不过可以沿着这个思路走：
先按模块归类，然后对模块内再进一步归类
模块 /文件是独立的，这个可以区分开，但是一个模块中可能会输出不同的日志，比如说方法 A 有多个地方会有异常栈输出，且每个异常栈的信息可能略有不同。行数信息是在日志的描述中的一个变量，所以没办法将行数作为直接标识。
不过可以沿着这个思路走：
先按模块归类，然后对模块内再进一步归类
2018-06-05 20:30:55 +08:00 回复了 shiznet 创建的主题 › 问与答 › 如何实现一个基于 pattern 的文本相似度聚类 |
@ipwx 看了下「 DeepLog: Anomaly Detection and Diagnosis from System Logs through Deep Learning 」摘要，感觉和我需求不大一致。
```
Anomaly detection is a critical step towards building a secure and
trustworthy system. The primary purpose of a system log is to
record system states and significant events at various critical points
to help debug system failures and perform root cause analysis. Such
log data is universally available in nearly all computer systems.
Log data is an important and valuable resource for understanding
system status and performance issues; therefore, the various system
logs are naturally excellent source of information for online
monitoring and anomaly detection. We propose DeepLog, a deep
neural network model utilizing Long Short-Term Memory (LSTM),
to model a system log as a natural language sequence. is allows
DeepLog to automatically learn log patterns from normal execution,
and detect anomalies when log patterns deviate from the model
trained from log data under normal execution. In addition, we
demonstrate how to incrementally update the DeepLog model in
an online fashion so that it can adapt to new log patterns over time.
Furthermore, DeepLog constructs workows from the underlying
system log so that once an anomaly is detected, users can diagnose
the detected anomaly and perform root cause analysis effectively.
Extensive experimental evaluations over large log data have shown
that DeepLog has outperformed other existing log-based anomaly
detection methods based on traditional data mining methodologies.
```
```
Anomaly detection is a critical step towards building a secure and
trustworthy system. The primary purpose of a system log is to
record system states and significant events at various critical points
to help debug system failures and perform root cause analysis. Such
log data is universally available in nearly all computer systems.
Log data is an important and valuable resource for understanding
system status and performance issues; therefore, the various system
logs are naturally excellent source of information for online
monitoring and anomaly detection. We propose DeepLog, a deep
neural network model utilizing Long Short-Term Memory (LSTM),
to model a system log as a natural language sequence. is allows
DeepLog to automatically learn log patterns from normal execution,
and detect anomalies when log patterns deviate from the model
trained from log data under normal execution. In addition, we
demonstrate how to incrementally update the DeepLog model in
an online fashion so that it can adapt to new log patterns over time.
Furthermore, DeepLog constructs workows from the underlying
system log so that once an anomaly is detected, users can diagnose
the detected anomaly and perform root cause analysis effectively.
Extensive experimental evaluations over large log data have shown
that DeepLog has outperformed other existing log-based anomaly
detection methods based on traditional data mining methodologies.
```
2018-01-04 07:31:09 +08:00 回复了 LeCent 创建的主题 › 程序员 › 如何统计一座城市的汽车数量？ |
人均汽车保有量*人数
2017-01-19 09:54:47 +08:00 回复了 jellybool 创建的主题 › PHP › 在我的背景音乐里，写代码没人能超过我 |
@starvedcat 话音没落就被对方 DJ 推下了悬崖
2015-11-19 22:22:57 +08:00 回复了 jukka 创建的主题 › 程序员 › 说说自己写过的最难懂的代码吧。 |
擦 忽略吧 文不对题
2015-11-19 22:21:57 +08:00 回复了 jukka 创建的主题 › 程序员 › 说说自己写过的最难懂的代码吧。 |
混淆后的代码 类名变量全是 abc
2015-10-03 18:43:23 +08:00 回复了 lirijie1 创建的主题 › 程序员 › 一个有意思的小问题，不知道有没有人碰到过？ |
怪物出现的顺序是可以自定的么？
2015-07-07 11:49:42 +08:00 回复了 yzn 创建的主题 › 程序员 › 请教防止恶意注册的方法，急！！！ |
限IP会误伤，有些办公网公用一个出口IP。
增加下行短信验证，这样会增加短信的费用，但是可以防止机器批量注册。短信验证时最好配合验证码，否则下行短信接口可能会被人用来做短信炸弹。
对于批量注册，id通常都是有规律的，可以查看日志，对于短时间内集中注册的有规律的用户名可以监控起来。
增加下行短信验证，这样会增加短信的费用，但是可以防止机器批量注册。短信验证时最好配合验证码，否则下行短信接口可能会被人用来做短信炸弹。
对于批量注册，id通常都是有规律的，可以查看日志，对于短时间内集中注册的有规律的用户名可以监控起来。
Select Language