Scribe 安装使用

[wsgzao]

## 前言

Scribe 是 Facebook 开源的日志收集工具，我建议大家还是老老实实使用 ELK 或者 Splunk，不要瞎折腾 Scribe

> Centos 安装 Scribe 真是累

## 更新历史

2018 年 09 月 27 日 - 初稿

阅读原文 - https://wsgzao.github.io/post/scribe/

**扩展阅读**

Scribe - https://github.com/facebookarchive/scribe
Thrift - http://thrift.apache.org/docs/install/centos

---

## Scribe 简介

Scribe is a server for aggregating log data that's streamed in real time from clients. It is designed to be scalable and reliable.ß

See the Scribe Wiki for documentation:
http://wiki.github.com/facebook/scribe

## install scribe

> OS: CentOS 7.x

This guide shows how to install the precompiled scribed binary and its dependencies on a new server. Care must be taken to use the required versions for `boost (1.44.0)`, `thrift (0.7.0)` and `libevent (1.4.10)`.


``` bash
# install necessary tools
sudo yum install git make flex bison libtool automake openssl-devel libevent libevent-devel python-devel gcc-c++ byacc java-1.7.0-openjdk ant

# update autoconf
cd ~
sudo rpm -e --nodeps `rpm -qf /usr/bin/autoconf`
wget http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz
tar xzf autoconf-2.69.tar.gz
cd autoconf-2.69/
./configure
make
sudo make install
# make sure /usr/local/bin is in system PATH: add following line to ~/.bashrc
vi ~/.bashrc
export PATH=$PATH:/usr/local/bin

# install boost
cd ~
wget http://jaist.dl.sourceforge.net/project/boost/boost/1.44.0/boost_1_44_0.tar.gz
tar xzf boost_1_44_0.tar.gz
cd boost_1_44_0/
./bootstrap.sh
sudo ./bjam install

# install thrift
cd ~
git clone https://github.com/apache/thrift.git
cd thrift/
git fetch
git branch -a
git checkout 0.7.x
./bootstrap.sh
./configure
make
cp compiler/cpp/thrifty.hh compiler/cpp/thrifty.h
make
sudo make install
cd lib/py/
sudo python setup.py install

# install fb303
cd ~/thrift/contrib/fb303/
./bootstrap.sh
./configure CPPFLAGS="-DHAVE_INTTYPES_H -DHAVE_NETINET_IN_H"
make
sudo make install
cd py/
sudo python setup.py install

# install libevent
cd ~
wget https://github.com/downloads/libevent/libevent/libevent-1.4.10-stable.tar.gz
tar xf libevent-1.4.10-stable.tar.gz
cd libevent-1.4.10-stable
./configure
make
make install

# Copy the scribed binary and put it in the desired directory
scp scribe.tar.gz 10.65.32.134:/tmp
scp scribed 10.65.32.134:/tm
# unzip and mkdir
mkdir /opt/running
cd /opt/running
cp /tmp/scribe.tar.gz .
tar xzvf scribe.tar.gz
rm -f scribe.tar.gz
mkdir -p /data/gop/live/primary/gop
mkdir -p /data/gop/live/secondary/gop
cp /tmp/scribed /usr/local/bin/scribed
cd scribe/

[root@vm4 scribe]# cat run_scribed.sh
#!/bin/bash

while true
do
export LD_LIBRARY_PATH=/usr/lib:/usr/local/lib
/usr/local/bin/scribed scribe_11315.conf 1>output/scribe.log 2>output/scribe.err
sleep 1
done

nohup sh run_scribed.sh &

[root@sg-gop-10-65-32-134 scribe]# netstat -tlnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:10050 0.0.0.0:* LISTEN 55310/zabbix_agentd
tcp 0 0 0.0.0.0:38422 0.0.0.0:* LISTEN 52781/sshd
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1305/sshd
tcp6 0 0 :::10050 :::* LISTEN 55310/zabbix_agentd
tcp6 0 0 :::11315 :::* LISTEN 4294/scribed
tcp6 0 0 :::38422 :::* LISTEN 52781/sshd
tcp6 0 0 :::22 :::* LISTEN 1305/sshd

```