Home Sign Up Sign In
推荐学习书目
› Learn Python the Hard Way
Python Sites
› PyPI - Python Package Index
› http://diveintopython.org/toc/index.html
› Pocoo
值得关注的项目
› PyPy
› Celery
› Jinja2
› Read the Docs
› gevent
› pyenv
› virtualenv
› Stackless Python
› Beautiful Soup
› 结巴中文分词
› Green Unicorn
› Sentry
› Shovel
› Pyflakes
› pytest
Python 编程
› pep8 Checker
Styles
› PEP 8
› Google Python Style Guide
› Code Style from The Hitchhiker's Guide
MrMike
V2EX  ›  Python

请教下 supervisor 配置的问题

  •   
  •   MrMike · Apr 15, 2017 · 5051 views
    This topic created in 3318 days ago, the information mentioned may be changed or developed.

    为了监控 elasticsearch 死掉后，能自动重启，网上查了下,supervisor 可以实现这个要求，以下是配置： [program=sticsearch] command=/usr/share/elasticsearch/bin/elasticsearch autostart=true autorestart=true stdout_logfile=/home/demoer/log/elasticsearch/elasticsearch-stdout.log stderr_logfile=/home/demoer/log/elasticsearch/elasticsearch-stderr.log startsecs=5

    我把这个直接写在 supervisor.conf ，在启动 supervisorctl reload, reread, update,都说没有找到配置（ No config updates to processes ）。 我用 include_file 单独来引入，也是同样的结果。 请问下配置代码是否正确呢？

  • log
  • demoer
  • Supervisor
  • Elasticsearch
    29 replies  •  2017-04-15 23:34:39 +08:00
    RIcter
        1
    RIcter  
       Apr 15, 2017 via iPhone
    得重启 supervisor 服务…
    MrMike
        2
    MrMike  
    OP
       Apr 15, 2017
    supervisord -c /etc/supervisord.conf

    [root tmp]# ps aux | grep supervisord
    root 10926 0.0 0.3 220120 11836 ? Ss Apr14 0:11 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
    root 12554 0.0 0.2 217092 10664 ? Ss Apr14 0:10 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
    root 12564 0.0 0.2 217128 10716 ? Ss Apr14 0:10 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
    root 12575 0.0 0.2 217280 10824 ? Ss Apr14 0:11 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
    root 12764 0.0 0.2 219728 11464 ? Ss Apr14 0:11 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
    root 13844 0.0 0.2 217588 11164 ? Ss 00:11 0:05 /usr/bin/python /usr/bin/supervisord
    root 14070 0.0 0.2 216964 9828 ? Ss 01:00 0:04 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
    root 14117 0.0 0.2 217144 10644 ? Ss 01:02 0:04 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
    root 14128 0.0 0.2 217520 11144 ? Ss 01:04 0:04 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
    root 14936 0.0 0.2 216960 9832 ? Ss 08:07 0:00 /usr/bin/python /usr/bin/supervisord -c /etc/supervisord.conf
    root 14938 0.0 0.0 112648 960 pts/0 S+ 08:07 0:00 grep --color=auto supervisord

    [root tmp]# supervisorctl
    supervisor> reload
    Really restart the remote supervisord process y/N? y
    Restarted supervisord
    supervisor> reread
    No config updates to processes
    supervisor> update
    supervisor> status
    supervisor>

    已经重启过很多次，还是一样的结果。

    下面是 supervisor.conf 和 elasticsearch.conf 的配置内容。

    supervisor.conf:

    [include]
    files = /etc/supervisord/conf.d/elasticsearch.conf

    elasticsearch.conf:

    [program=sticsearch]
    command=systemctl restart elasticsearch.service
    ;command=/usr/share/elasticsearch/bin/elasticsearch
    ;directory=/usr/share/elasticsearch
    autostart=true
    autorestart=true
    startsecs=10
    startretries=3
    redirect_stderr=true
    user=root
    stdout_logfile=/home/public/log/elasticsearch/elasticsearch-stdout.log
    stderr_logfile=/home/public/log/elasticsearch/elasticsearch-stderr.log
    zzlyzq
        3
    zzlyzq  
       Apr 15, 2017 via iPhone
    主配置里面默认会包含所有 conf.d 下面的 conf 后缀文件 回头我看看我的配置
    MrMike
        4
    MrMike  
    OP
       Apr 15, 2017
    @zzlyzq 我安装完 supervisor 后，在 etc 下面没有 /etc/supervisord 文件夹的，我是自己创建的 /etc/supervisord/con.d/文件夹，然后把 elasticsearch.conf 放在里面。
    zzlyzq
        5
    zzlyzq  
       Apr 15, 2017
    恩 看起来没啥错，我的 centos6.8 安装后，主文件是 包括如下两行

    [include]
    files = /etc/supervisor.d/*.conf

    你的配置看起来也没啥问题的，要不试试
    ls /etc/supervisord/conf.d/elasticsearch.conf
    看看这个文件是不是真正的存在呢？ 不存在的概率可能比较小，不过工作中经常会遇到这种情况，呵呵。
    MrMike
        6
    MrMike  
    OP
       Apr 15, 2017
    @zzlyzq 文件是存在的。郁闷。。一直找不到原因。
    zzlyzq
        7
    zzlyzq  
       Apr 15, 2017
    @MrMike 既然文件已经加载了，怀疑还是配置的问题，
    1. 是不是配置的 program 名字有冲突？
    2. 打开界面，可能更一目了然

    在 supervisord.conf 开头加上或者修改如下，打开 http ，一目了然

    [unix_http_server]
    file=/tmp/supervisor.sock ; (the path to the socket file)
    [inet_http_server] ; inet (TCP) server disabled by default
    port=0.0.0.0:1009 ; (ip_address:port specifier, *:port for all iface)
    username=ops ; (default is no username (open server))
    password=123123 ; (default is no password (open server))
    zzlyzq
        8
    zzlyzq  
       Apr 15, 2017
    @MrMike 我大概知道了， supervisor 在执行命令的时候要求比较苛刻，比如全路径呀，环境变量报错呀啥的， 刚刚的 http 页面可以让你看到
    1. elasticsearch 进程是否配置到 supervisord
    2. 启动情况如何，是 ok 还是 fail
    3. 瞅瞅 log 里面有啥吧
    MrMike
        9
    MrMike  
    OP
       Apr 15, 2017
    @zzlyzq supervisord.conf 用的是默认设置，我没修过过。

    [unix_http_server]
    file=/tmp/supervisor.sock ; (the path to the socket file)
    chmod=0777 ; socket file mode (default 0700)
    ;chown=nobody:nogroup ; socket file uid:gid owner
    ;username=user ; (default is no username (open server))
    ;password=123 ; (default is no password (open server))

    ;[inet_http_server] ; inet (TCP) server disabled by default
    ;port=127.0.0.1:9001 ; (ip_address:port specifier, *:port for all iface)
    ;username=user ; (default is no username (open server))
    ;password=123 ; (default is no password (open server))

    supervisord.log

    2017-04-15 08:08:21,706 INFO RPC interface 'supervisor' initialized
    2017-04-15 08:08:21,706 CRIT Server 'unix_http_server' running without any HTTP authentication checking
    2017-04-15 08:08:21,707 INFO supervisord started with pid 14936
    2017-04-15 08:43:04,969 CRIT Supervisor running as root (no user in config file)
    2017-04-15 08:43:04,969 INFO Included extra file "/etc/supervisord/conf.d/elasticsearch.conf" during parsing
    2017-04-15 08:43:04,980 INFO RPC interface 'supervisor' initialized
    2017-04-15 08:43:04,980 CRIT Server 'unix_http_server' running without any HTTP authentication checking
    2017-04-15 08:43:04,981 INFO daemonizing the supervisord process
    2017-04-15 08:43:04,981 INFO supervisord started with pid 15044

    貌似没看出有啥问题呢。也没有报错。
    zzlyzq
        10
    zzlyzq  
       Apr 15, 2017
    Included extra file "/etc/supervisord/conf.d/elasticsearch.conf" during parsing

    上面的一行说明你的 elasticsearch.conf 已经被加载了，现在我越来越猜测是 elasticsearch.conf 配置问题导致没有启动

    既然没有打开 http ，那就使用如下命令看看 status 吧

    supervisorctl status
    zsz
        11
    zsz  
       Apr 15, 2017 via iPhone
    1.supervisor 配置中的 command 一定要处于挂起运行，把标准输出，是他的子进程，运行期间不能返回
    2.你需要用命令行启动 es 的方式，配置到 command
    3.既然用 systemd 和 supervisor 用一个就可以，他们都是守护


    有问题可以加群问，效率高些，面向初学者的 python Linux 学习群， 由一群有经验的工程师组成， python qq 群号： 278529278 ， PHP qq 群号： 476648701 ，非商业性质，拒绝广告，只接收真正想学这方面技术的朋友，交流学习，申请请说明来自 v2ex
    yzongyue
        12
    yzongyue  
       Apr 15, 2017
    这样写试试 [program:sticsearch]
    ksupertu
        13
    ksupertu  
       Apr 15, 2017 via iPhone
    新版本先要启动 supervisord 然后在启动 supervisorctl ，不然会一直报错……
    ksupertu
        14
    ksupertu  
       Apr 15, 2017 via iPhone
    而且你 command 那里最好写成 shell 脚本去执行，错误概率小一点
    MrMike
        15
    MrMike  
    OP
       Apr 15, 2017
    @ksupertu 已经启动过了。 command 我用 /usr/share/elasticsearch/bin/elasticsearch 也是不能用的。在日志里面也看不到错误提示呢。
    zsz
        16
    zsz  
       Apr 15, 2017 via iPhone
    @MrMike ，你看的什么错误日志，是 supervisor 的？还是 supervisor 配置文件中 es 的？这个启动不了应该是环境问题，一般是 java 环境， supervisor 自己的错误日志中应该有信息才对
    MrMike
        17
    MrMike  
    OP
       Apr 15, 2017
    @zsz 是 supervisord.log
    a128445
        18
    a128445  
       Apr 15, 2017
    [program=sticsearch]

    改为
    [program:sticsearch]
    zsz
        19
    zsz  
       Apr 15, 2017
    @MrMike 这两个 log 呢？ 有错误信息

    stdout_logfile=/home/public/log/elasticsearch/elasticsearch-stdout.log
    stderr_logfile=/home/public/log/elasticsearch/elasticsearch-stderr.log
    zsz
        20
    zsz  
       Apr 15, 2017
    先按照上面说的把配置写对 [program:sticsearch]
    MrMike
        21
    MrMike  
    OP
       Apr 15, 2017
    @zsz
    @a128445

    elasticsearch.con ：

    [program:elasticsearch]
    command=/usr/share/elasticsearch/bin/elasticsearch
    directory=/usr/share/elasticsearch
    autostart=true
    autorestart=true
    startsecs=10
    startretries=3
    redirect_stderr=true
    user=root
    stdout_logfile=/home/demoer/log/elasticsearch/elasticsearch-stdout.log
    stderr_logfile=/home/demoer/log/elasticsearch/elasticsearch-stderr.log


    elasticsearch-stdout.log ：

    Exception in thread "main" ElasticsearchException[Failed to load logging configuration]; nested: NoSuchFileException[/usr/share/elasticsearch/config];
    Likely root cause: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/config
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
    at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
    at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
    at java.nio.file.Files.readAttributes(Files.java:1737)
    at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:225)
    at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
    at java.nio.file.Files.walkFileTree(Files.java:2662)
    at org.elasticsearch.common.logging.LogConfigurator.resolveConfig(LogConfigurator.java:150)
    at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:114)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:236)
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:96)
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:91)
    at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54)
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:91)
    at org.elasticsearch.cli.Command.main(Command.java:53)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:70)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:63)
    Refer to the log for complete error details.
    Exception in thread "main" ElasticsearchException[Failed to load logging configuration]; nested: NoSuchFileException[/usr/share/elasticsearch/config];
    Likely root cause: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/config
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
    at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
    at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
    at java.nio.file.Files.readAttributes(Files.java:1737)
    at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:225)
    at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
    at java.nio.file.Files.walkFileTree(Files.java:2662)
    at org.elasticsearch.common.logging.LogConfigurator.resolveConfig(LogConfigurator.java:150)
    at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:114)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:236)
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:96)
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:91)
    at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54)
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:91)
    at org.elasticsearch.cli.Command.main(Command.java:53)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:70)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:63)
    Refer to the log for complete error details.
    Exception in thread "main" ElasticsearchException[Failed to load logging configuration]; nested: NoSuchFileException[/usr/share/elasticsearch/config];
    Likely root cause: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/config
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
    at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
    at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
    at java.nio.file.Files.readAttributes(Files.java:1737)
    at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:225)
    at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
    at java.nio.file.Files.walkFileTree(Files.java:2662)
    at org.elasticsearch.common.logging.LogConfigurator.resolveConfig(LogConfigurator.java:150)
    at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:114)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:236)
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:96)
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:91)
    at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54)
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:91)
    at org.elasticsearch.cli.Command.main(Command.java:53)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:70)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:63)
    Refer to the log for complete error details.
    Exception in thread "main" ElasticsearchException[Failed to load logging configuration]; nested: NoSuchFileException[/usr/share/elasticsearch/config];
    Likely root cause: java.nio.file.NoSuchFileException: /usr/share/elasticsearch/config
    at sun.nio.fs.UnixException.translateToIOException(UnixException.java:86)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
    at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
    at sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(UnixFileAttributeViews.java:55)
    at sun.nio.fs.UnixFileSystemProvider.readAttributes(UnixFileSystemProvider.java:144)
    at sun.nio.fs.LinuxFileSystemProvider.readAttributes(LinuxFileSystemProvider.java:99)
    at java.nio.file.Files.readAttributes(Files.java:1737)
    at java.nio.file.FileTreeWalker.getAttributes(FileTreeWalker.java:225)
    at java.nio.file.FileTreeWalker.visit(FileTreeWalker.java:276)
    at java.nio.file.FileTreeWalker.walk(FileTreeWalker.java:322)
    at java.nio.file.Files.walkFileTree(Files.java:2662)
    at org.elasticsearch.common.logging.LogConfigurator.resolveConfig(LogConfigurator.java:150)
    at org.elasticsearch.common.logging.LogConfigurator.configure(LogConfigurator.java:114)
    at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:236)
    at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:96)
    at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:91)
    at org.elasticsearch.cli.SettingCommand.execute(SettingCommand.java:54)
    at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:91)
    at org.elasticsearch.cli.Command.main(Command.java:53)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:70)
    at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:63)
    Refer to the log for complete error details.


    elasticsearch-stderr.log 内容为空。

    我在重启 supervisor 后，进入 supervisorctl ，提示“ elasticsearch FATAL Exited too quickly (process log may have details)”
    MrMike
        22
    MrMike  
    OP
       Apr 15, 2017
    elasticsearch 安装后，在 /usr/share/elasticsearch/就没有 config 这个文件的。
    zsz
        23
    zsz  
       Apr 15, 2017 via iPhone
    1. 你命令行成功启动过 es 么？
    2.如果你只是在 systemd 中成功启动过，你的配置文件应该在系统级的目录下，找到他，先用命令行能启动，再把命令配置进去， es 在命令行可以指定配置文件的
    MrMike
        24
    MrMike  
    OP
       Apr 15, 2017
    @zsz ES 是可以正常启动和关闭的。我再看看这个错误提示到底是咋回事。。
    MrMike
        25
    MrMike  
    OP
       Apr 15, 2017
    @zsz 我试试用命令行启动下。
    MrMike
        26
    MrMike  
    OP
       Apr 15, 2017
    @zsz
    @a128445

    配置文件改成这样的：

    [program:elasticsearch]
    command=/etc/init.d/elasticsearch start
    directory=/usr/share/elasticsearch
    autostart=true
    autorestart=true
    startsecs=10
    startretries=3
    redirect_stderr=true
    user=root
    stdout_logfile=/home/demoer/log/elasticsearch/elasticsearch-stdout.log
    stderr_logfile=/home/demoer/log/elasticsearch/elasticsearch-stderr.log

    我手动关闭 es 后，等了几秒钟 es 没有启动，就使用 supervisorctl restart elasticsearch 启动，
    elasticsearch: ERROR (not running)
    elasticsearch: ERROR (spawn error)

    得到这样的错误提示，但是 es 却重新启动了。不能自动启动 es 呢。。
    MrMike
        27
    MrMike  
    OP
       Apr 15, 2017
    @zsz stop es 后，我目前的配置，只能手动执行 supervisorctl reload ，才能再让 es 启动。
    zsz
        28
    zsz  
       Apr 15, 2017 via iPhone
    1. 上面我指出过 supervisor 中 command 不能使用 systemctl 或者 init.d 的方式，因为这两种是 daemon 运行，命令执行完就返回了
    2. 你加我上面的发群我用 qq 跟你说，这样解释太慢了，非商业化的群，不用担心
    MrMike
        29
    MrMike  
    OP
       Apr 15, 2017
    @zsz 加了，回你信息了
    About   ·   Help   ·   Advertise   ·   Blog   ·   API   ·   FAQ   ·   Solana   ·   2933 Online   Highest 6679   ·     Select Language
    创意工作者们的社区
    World is powered by solitude
    VERSION: 3.9.8.5 · 133ms · UTC 09:46 · PVG 17:46 · LAX 02:46 · JFK 05:46
    ♥ Do have faith in what you're doing.