文章 78
评论 0
浏览 8367
3-tomcat会话(session)复制集群

3-tomcat会话(session)复制集群

3 Tomcat Session Replication Cluster

Tomcat 官方实现了 Session 的复制集群,将每个Tomcat的Session进行相互的复制同步,从而保证所有Tomcat都有相同的Session信息.

clipboard.png

3.1 配置说明

官方文档:https://tomcat.apache.org/tomcat-8.5-doc/cluster-howto.html

<Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
                 channelSendOptions="8">

          <Manager className="org.apache.catalina.ha.session.DeltaManager"
                   expireSessionsOnShutdown="false"
                   notifyListenersOnReplication="true"/>

          <Channel className="org.apache.catalina.tribes.group.GroupChannel">
            <Membership className="org.apache.catalina.tribes.membership.McastService"
                        address="228.0.0.4"    #指定的多播地址
                        port="45564"           #45564/UDP
                        frequency="500"        #间隔500ms发送
                        dropTime="3000"/>
            <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                      address="auto"           #监听地址,此项建议修改为当前主机的IP
                      port="4000"              #监听端口
                      autoBind="100"           #如果端口冲突,自动绑定其它端口,范围是4000-4100
                      selectorTimeout="5000"    #自动绑定超时时长5s
                      maxThreads="6"/>

            <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
              <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
            </Sender>
            <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
            <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>
          </Channel>

          <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
                 filter=""/>
          <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

          <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
                    tempDir="/tmp/war-temp/"
                    deployDir="/tmp/war-deploy/"
                    watchDir="/tmp/war-listen/"
                    watchEnabled="false"/>

          <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
        </Cluster>
#注意:tomcat7的官方文档此处有错误
http://tomcat.apache.org/tomcat-7.0-doc/cluster-howto.html
......
<ClusterListener
className="org.apache.catalina.ha.session.JvmRouteSessionIDBinderListener">
        <ClusterListener
className="org.apache.catalina.ha.session.ClusterSessionListener">
      </Cluster>

配置说明:

  • Cluster 集群配置

  • Manager 会话管理器配置

  • Channel 信道配置

    • Membership 成员判定。使用什么多播地址、端口多少、间隔时长ms、超时时长ms。同一个多播地址和端口认为同属一个组。使用时修改这个多播地址,以防冲突
    • Receiver 接收器,多线程接收多个其他节点的心跳、会话信息。默认会从4000到4100依次尝试可用端口
      • address="auto",auto可能绑定到127.0.0.1上,所以一定要改为当前主机可用的IP
    • Sender 多线程发送器,内部使用了tcp连接池。
    • Interceptor 拦截器
  • Valve

    • ReplicationValve 检测哪些请求需要检测Session,Session数据是否有了变化,需要启动复制过程
  • ClusterListener

    • ClusterSessionListener 集群session侦听器

使用

添加到 所有虚拟主机都可以启用Session复制

添加到 ,该虚拟主机可以启用Session复制

最后,在应用程序内部启用了才可以使用

3.2 实战案例: 实现 Tomcat Session 集群

clipboard.png

环境准备:

  • 时间同步,确保NTP或Chrony服务正常运行
  • 防火墙规则

3.2.1 在 proxy 主机设置 httpd (或nginx)实现后端tomcat主机轮询

[14:23:20 root@proxy ~]#cat /etc/httpd/conf.d/tomcat.conf
#Header add Set-Cookie "ROUTEID=.%{BALANCER_WORKER_ROUTE}e; path=/" env=BALANCER_ROUTE_CHANGED


<Proxy balancer://tomcat-server>
    BalancerMember http://t1.zhangzhuo.org:8080 loadfactor=1 route=T1
    BalancerMember http://t2.zhangzhuo.org:8080 loadfactor=1 route=T2
    #ProxySet stickysession=ROUTEID
</Proxy>

<VirtualHost *:80>
    ServerName    proxy.zhangzhuo.org
    ProxyRequests Off
    ProxyVia      On
    ProxyPreserveHost On
    ProxyPass / balancer://tomcat-server/
    ProxyPassReverse / balancer://tomcat-server/
</VirtualHost>

<Location /balancer-manager>
    SetHandler balancer-manager
    ProxyPass !
    Require all granted
</Location>
[14:23:54 root@proxy ~]#systemctl restart httpd

3.2.2 在所有后端tomcat主机上修改conf/server.xml

本次把多播复制的配置放到t1.magedu.org和t2.magedu.org虚拟主机里面, 即Host块中。

特别注意修改Receiver的address属性为一个本机可对外的IP地址

3.2.2.1 修改 t1 主机的 conf/server.xml

#将5.1 内容复制到conf/server.xml的Host块内或Engine块(针对所有主机)
[14:28:29 root@t1 ~]#vim /usr/local/tomcat/conf/server.xml 
            <Host name="t1.zhangzhuo.org"  appBase="/data/webapps"
            unpackWARs="true" autoDeploy="true">

       <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
                 channelSendOptions="8">

          <Manager className="org.apache.catalina.ha.session.DeltaManager"
                   expireSessionsOnShutdown="false"
                   notifyListenersOnReplication="true"/>

          <Channel className="org.apache.catalina.tribes.group.GroupChannel">
            <Membership className="org.apache.catalina.tribes.membership.McastService"
                        address="230.100.100.100"
                        port="45564"
                        frequency="500"
                        dropTime="3000"/>
            <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                      address="192.168.10.82"
                      port="4000"
                      autoBind="100"
                      selectorTimeout="5000"
                      maxThreads="6"/>

            <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
              <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
            </Sender>
            <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
            <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>
          </Channel>

          <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
                 filter=""/>
          <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

          <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
                    tempDir="/tmp/war-temp/"
                    deployDir="/tmp/war-deploy/"
                    watchDir="/tmp/war-listen/"
                    watchEnabled="false"/>

          <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
        </Cluster>
[14:29:07 root@t1 ~]#ss -ntl
State    Recv-Q   Send-Q               Local Address:Port       Peer Address:Port   
LISTEN   0        128                        0.0.0.0:111             0.0.0.0:*      
LISTEN   0        128                        0.0.0.0:22              0.0.0.0:*      
LISTEN   0        128                           [::]:111                [::]:*      
LISTEN   0        100                              *:8080                  *:*      
LISTEN   0        128                           [::]:22                 [::]:*      
LISTEN   0        50          [::ffff:192.168.10.82]:4000                  *:*      
LISTEN   0        100                              *:8009                  *:*

简化说明

t1的conf/server.xml中,如下
<Host name="t1.magedu.com" appBase="/data/webapps" autoDeploy="true" >
  #其他略去
   <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
 address="192.168.10.82"  #只改此行
 port="4000"
 autoBind="100"
 selectorTimeout="5000"
 maxThreads="6"/>

3.2.2.2 修改 t2 主机的 conf/server.xml

[11:41:50 root@t2 ~]#vim /usr/local/tomcat/conf/server.xml
     <Cluster className="org.apache.catalina.ha.tcp.SimpleTcpCluster"
                 channelSendOptions="8">

          <Manager className="org.apache.catalina.ha.session.DeltaManager"
                   expireSessionsOnShutdown="false"
                   notifyListenersOnReplication="true"/>

          <Channel className="org.apache.catalina.tribes.group.GroupChannel">
            <Membership className="org.apache.catalina.tribes.membership.McastService"
                        address="230.100.100.100"
                        port="45564"
                        frequency="500"
                        dropTime="3000"/>
            <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
                      address="192.168.10.83"    #此行指定当前主机的IP,其它和T1节点配置相同
                      port="4000"
                      autoBind="100"
                      selectorTimeout="5000"
                      maxThreads="6"/>

            <Sender className="org.apache.catalina.tribes.transport.ReplicationTransmitter">
              <Transport className="org.apache.catalina.tribes.transport.nio.PooledParallelSender"/>
            </Sender>
            <Interceptor className="org.apache.catalina.tribes.group.interceptors.TcpFailureDetector"/>
            <Interceptor className="org.apache.catalina.tribes.group.interceptors.MessageDispatchInterceptor"/>
          </Channel>

          <Valve className="org.apache.catalina.ha.tcp.ReplicationValve"
                 filter=""/>
          <Valve className="org.apache.catalina.ha.session.JvmRouteBinderValve"/>

          <Deployer className="org.apache.catalina.ha.deploy.FarmWarDeployer"
                    tempDir="/tmp/war-temp/"
                    deployDir="/tmp/war-deploy/"
                    watchDir="/tmp/war-listen/"
                    watchEnabled="false"/>

          <ClusterListener className="org.apache.catalina.ha.session.ClusterSessionListener"/>
        </Cluster>
[14:32:37 root@t2 ~]#ss -ntl
State    Recv-Q   Send-Q               Local Address:Port       Peer Address:Port   
LISTEN   0        128                        0.0.0.0:111             0.0.0.0:*      
LISTEN   0        128                        0.0.0.0:22              0.0.0.0:*      
LISTEN   0        100                              *:8009                  *:*      
LISTEN   0        128                           [::]:111                [::]:*      
LISTEN   0        100                              *:8080                  *:*      
LISTEN   0        128                           [::]:22                 [::]:*      
LISTEN   0        50          [::ffff:192.168.10.83]:4000                  *:*

简化说明

t2主机的server.xml中,如下

<Host name="t2.magedu.com" appBase="/data/webapps" autoDeploy="true" >
  其他略去
   <Receiver className="org.apache.catalina.tribes.transport.nio.NioReceiver"
 address="192.168.10.83"  #只改此行
 port="4000"
 autoBind="100"
 selectorTimeout="5000"
 maxThreads="6"/>

尝试使用刚才配置过得负载均衡(移除Session黏性),测试发现Session还是变来变去。

3.2.3 修改应用的web.xml文件开启该应用程序的分布式

参考官方说明: https://tomcat.apache.org/tomcat-8.5-doc/cluster-howto.html
clipboard.png

Make sure your web.xml has the  element

为所有tomcat主机应用web.xml的 标签增加子标签 来开启该应用程序的分布式。

3.2.3.1 修改t1主机的应用的web.xml文件

[14:29:07 root@t1 ~]#ll /usr/local/tomcat/webapps/ROOT/WEB-INF/
total 4
-rw-r----- 1 tomcat tomcat 1227 Mar  5 07:17 web.xml
[14:37:39 root@t1 ~]#cp -a /usr/local/tomcat/webapps/ROOT/WEB-INF /data/webapps/ROOT/
[14:38:01 root@t1 ~]#tree /data/webapps/ROOT/
/data/webapps/ROOT/
├── index.jsp
└── WEB-INF
    └── web.xml
#在倒数第二行加一行
[14:38:16 root@t1 ~]#vim /data/webapps/ROOT/WEB-INF/web.xml
  </description>
<distributable/>   #添加此行
</web-app>
#注意权限
[14:39:26 root@t1 ~]#ll /data/webapps/ROOT/WEB-INF/
total 4
-rw-r----- 1 tomcat tomcat 1244 Mar 18 14:39 web.xml
[14:41:28 root@t1 ~]#systemctl restart tomcat.service
#同时观察日志
[14:43:59 root@t1 ~]#systemctl restart tomcat.service
18-Mar-2021 14:41:54.616 INFO [Membership-MemberAdded.] org.apache.catalina.ha.tcp.SimpleTcpCluster.memberAdded Replication member added:[org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 10, 83}:4000,{192, 168, 10, 83},4000, alive=559883, securePort=-1, UDP Port=-1, id={87 114 -110 73 126 -34 64 126 -77 51 -24 81 -58 55 116 47 }, payload={}, command={}, domain={}]]

3.2.3.2 修改t2主机的应用的web.xml文件

#与5.2.3.1上的t1相同的操作
[14:32:37 root@t2 ~]#cp -a /usr/local/tomcat/webapps/ROOT/WEB-INF /data/webapps/ROOT/
[14:43:07 root@t2 ~]#vim /data/webapps/ROOT/WEB-INF/web.xml 
  </description>
<distributable/> #添加此行
</web-app>
#注意权限
[14:43:28 root@t2 ~]#ll /data/webapps/ROOT/WEB-INF/web.xml
-rw-r----- 1 tomcat tomcat 1244 Mar 18 14:43 /data/webapps/ROOT/WEB-INF/web.xml
#同时观察日志
[14:43:59 root@t2 ~]#systemctl restart tomcat.service
18-Mar-2021 14:44:49.868 INFO [Membership-MemberAdded.] org.apache.catalina.ha.tcp.SimpleTcpCluster.memberAdded Replication member added:[org.apache.catalina.tribes.membership.MemberImpl[tcp://{192, 168, 10, 82}:4000,{192, 168, 10, 82},4000, alive=175284, securePort=-1, UDP Port=-1, id={-49 -105 84 52 28 -34 64 -38 -115 -100 57 -77 83 32 14 -124 }, payload={}, command={}, domain={}]]

3.2.4 测试访问

重启全部Tomcat,通过负载均衡调度到不同节点,返回的SessionID不变了。

用浏览器访问,并刷新多次,发现SessionID 不变,但后端主机在轮询

但此方式当后端tomcat主机较多时,会重复占用大量的内存,并不适合后端服务器众多的场景

3.2.5 故障模拟

#模拟t2节点故障
[14:44:47 root@t2 ~]#systemctl stop tomcat.service
#多次访问SessionID不变

标题:3-tomcat会话(session)复制集群
作者:Carey
地址:HTTPS://zhangzhuo.ltd/articles/2021/03/20/1616229768971.html

生而为人

取消