/images/hugo/avatar.png

23.2 nginx基础入门

nginx基础入门

/images/linux_mt/linux_mt1.jpg

在学习 nginx 之前,我们首先来对 nginx 做一个入门介绍,后续我们会详细介绍 nginx web server 的配置。本节内容包括:

  1. nginx 框架
  2. nginx 安装
  3. nginx 配置文件格式

1. nginx 架构与特性

1.1 架构

/images/linux_mt/nginx.jpg

23.1 IO模型

IO模型

/images/linux_mt/linux_mt1.jpg

nginx 是一个 web 服务器,同时还能作为 http 协议的反向代理服务器。相比于 http,nginx 使用了更先进的 IO 模型,异步通信以及进程间通信的技术,能支持更多的并发请求,具有更高的性能和稳定性。本章我们首先来学习如何使用 nginx 配置一个 web serve,nginx 的反向代理功能我们留在 28 章再来介绍。本章内容包括

virtualenv

1. 环境创建

virtualenv dirname – 创建虚拟环境
source dirname/bin/activate – 启用虚拟环境

virtualenv 可用选项 作用
–distribute dirname 创建新的虚拟环境,并安装 pip
–no-site-packages 使系统环境的包对虚拟环境不可见

2.virtualenvwrapper

作用:virtualenv 管理工具,方便的创建/激活/管理/销毁虚拟环境

supervisor tornado 部署

1. tornado 启动

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
from tornado.netutil import set_close_exec


def main():
    app = AnalyticApiApplication()
    http_serve = httpserver.HTTPServer(app)
    # http_serve.listen(options.port)
    # supervisor 创建的监听套接字文件描述符,通过 0 号传递给 tornado的所有进程
    sock = socket.fromfd(0, family=socket.AF_INET, type=socket.SOCK_STREAM)
    set_close_exec(sock.fileno())
    sock.setblocking(0)   # 设置套接字为非阻塞调用
    http_serve.add_socket(sock)
    ioloop.IOLoop.instance().start()

2. supervisor 配置

1
2
3
4
5
6
7
8
command=/home/tao/.local/bin/pipenv run python app.py --connect=local-dev --debug=1
socket=tcp://localhost:8888
directory=/home/tao/projects/analytics_api
user=tao
numprocs=4
process_name=%(program_name)s_%(process_num)02d
stdout_logfile =/var/log/tornado_pyapi_stdout_%(process_num)02d.log  
stderr_logfile =/var/log/tornado_pyapi_stderr_%(process_num)02d.log

3. tornado.bind_socket

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
def bind_sockets(port, address=None, family=socket.AF_UNSPEC,
                 backlog=_DEFAULT_BACKLOG, flags=None, reuse_port=False):
    """Creates listening sockets bound to the given port and address.

    Returns a list of socket objects (multiple sockets are returned if
    the given address maps to multiple IP addresses, which is most common
    for mixed IPv4 and IPv6 use).

    Address may be either an IP address or hostname.  If it's a hostname,
    the server will listen on all IP addresses associated with the
    name.  Address may be an empty string or None to listen on all
    available interfaces.  Family may be set to either `socket.AF_INET`
    or `socket.AF_INET6` to restrict to IPv4 or IPv6 addresses, otherwise
    both will be used if available.

    The ``backlog`` argument has the same meaning as for
    `socket.listen() <socket.socket.listen>`.

    ``flags`` is a bitmask of AI_* flags to `~socket.getaddrinfo`, like
    ``socket.AI_PASSIVE | socket.AI_NUMERICHOST``.

    ``reuse_port`` option sets ``SO_REUSEPORT`` option for every socket
    in the list. If your platform doesn't support this option ValueError will
    be raised.
    """
    if reuse_port and not hasattr(socket, "SO_REUSEPORT"):
        raise ValueError("the platform doesn't support SO_REUSEPORT")

    sockets = []
    if address == "":
        address = None
    if not socket.has_ipv6 and family == socket.AF_UNSPEC:
        # Python can be compiled with --disable-ipv6, which causes
        # operations on AF_INET6 sockets to fail, but does not
        # automatically exclude those results from getaddrinfo
        # results.
        # http://bugs.python.org/issue16208
        family = socket.AF_INET
    if flags is None:
        flags = socket.AI_PASSIVE
    bound_port = None
    for res in set(socket.getaddrinfo(address, port, family, socket.SOCK_STREAM,
                                      0, flags)):
        af, socktype, proto, canonname, sockaddr = res
        if (sys.platform == 'darwin' and address == 'localhost' and
                af == socket.AF_INET6 and sockaddr[3] != 0):
            # Mac OS X includes a link-local address fe80::1%lo0 in the
            # getaddrinfo results for 'localhost'.  However, the firewall
            # doesn't understand that this is a local address and will
            # prompt for access (often repeatedly, due to an apparent
            # bug in its ability to remember granting access to an
            # application). Skip these addresses.
            continue
        try:
            sock = socket.socket(af, socktype, proto)
        except socket.error as e:
            if errno_from_exception(e) == errno.EAFNOSUPPORT:
                continue
            raise
        set_close_exec(sock.fileno())
        if os.name != 'nt':
            sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        if reuse_port:
            sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
        if af == socket.AF_INET6:
            # On linux, ipv6 sockets accept ipv4 too by default,
            # but this makes it impossible to bind to both
            # 0.0.0.0 in ipv4 and :: in ipv6.  On other systems,
            # separate sockets *must* be used to listen for both ipv4
            # and ipv6.  For consistency, always disable ipv4 on our
            # ipv6 sockets and use a separate ipv4 socket when needed.
            #
            # Python 2.x on windows doesn't have IPPROTO_IPV6.
            if hasattr(socket, "IPPROTO_IPV6"):
                sock.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 1)

        # automatic port allocation with port=None
        # should bind on the same port on IPv4 and IPv6
        host, requested_port = sockaddr[:2]
        if requested_port == 0 and bound_port is not None:
            sockaddr = tuple([host, bound_port] + list(sockaddr[2:]))

        sock.setblocking(0)
        sock.bind(sockaddr)
        bound_port = sock.getsockname()[1]
        sock.listen(backlog)
        sockets.append(sock)
    return sockets

16 wrapt 模块实战

装饰器和 wrapt 模块的介绍已经结束,作为整个系列的最后一篇的实战篇,我们来实现在一开始我提出的一个需求

1. 应用场景

在我日常的开发过程中,经常要查询各种数据库,比如 mysql, mongo,es 。基本上所有的数据库对查询语句能添加的查询条件都有限制。对于大批量的查询条件,只能分批多次查询,然后将查询结果合并。我们能不能将这种查询分批在合并的操作抽象出来实现为一个装饰器,在需要时对查询函数包装即可?下面是我的一个实现示例。

15 wrapt 模块使用

GrahamDumpleton wrapt blog 的翻译部分到此就结束。很可惜的是作者并没有把猴子补丁部分写完,查阅了 wrapt 的官方文档,上面只介绍了 wrapt 的装饰器,代理对象以及 synchronized 同步装饰器,也没有介绍猴子补丁相关内容。不过已经介绍的内容足够用了,接下来我想结合 wrapt 的文档介绍一下 wrapt 模块的使用,算是整个博客的总结。