This is a short story of fixing Tomcat in an EMR cluster. There is a CVE about Tomcat 8 hole. I’m using EMR 5.12.2. Let’s try installing CVE patch as a bootstrap action with
1 2 3 |
sudo yum update -y --releasever=latest --cve CVE-2019-0232 sudo shutdown -r now |
It should fix the mentioned hole but after bringing up the new cluster and running
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 |
[hadoop@ip-10-0-1-84 ~]$ yum info tomcat8 Loaded plugins: priorities, update-motd, upgrade-helper 6 packages excluded due to repository priority protections Installed Packages Name : tomcat8 Arch : noarch Version : 8.5.29 Release : 1.77.amzn1 Size : 324 k Repo : installed From repo : amzn-updates Summary : Apache Servlet/JSP Engine, RI for Servlet 3.1/JSP 2.3 API URL : http://tomcat.apache.org/ License : ASL 2.0 Description : Tomcat is the servlet container that is used in the official Reference : Implementation for the Java Servlet and JavaServer Pages technologies. : The Java Servlet and JavaServer Pages specifications are developed by : Sun under the Java Community Process. : : Tomcat is developed in an open and participatory environment and : released under the Apache Software License version 2.0. Tomcat is intended : to be a collaboration of the best-of-breed developers from around the world. |
we can see that it installs version 8.5.29 of tomcat (instead of 8.5.42 or something newer). Why? Let’s SSH into the EMR and check:
1 2 3 4 5 6 7 8 9 10 |
[hadoop@ip-10-0-1-84 ~]$ yum --showduplicates list tomcat8 Loaded plugins: priorities, update-motd, upgrade-helper 6 packages excluded due to repository priority protections Installed Packages tomcat8.noarch 8.5.29-1.77.amzn1 @amzn-updates Available Packages tomcat8.noarch 8.5.16-1.74.amzn1 amzn-main tomcat8.noarch 8.5.23-1.75.amzn1 amzn-updates tomcat8.noarch 8.5.28-1.76.amzn1 amzn-updates tomcat8.noarch 8.5.29-1.77.amzn1 amzn-updates |
So there is no newer version. But if we try applying the CVE patch again we get:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 |
[hadoop@ip-10-0-1-84 /]$ sudo yum update -y --releasever=latest --cve CVE-2019-0232 Loaded plugins: priorities, update-motd, upgrade-helper amzn-main | 2.1 kB 00:00:00 amzn-updates | 2.5 kB 00:00:00 emr-applications | 2.9 kB 00:00:00 8 packages excluded due to repository priority protections 6 package(s) needed (+0 related) for security, out of 130 available Resolving Dependencies --> Running transaction check ---> Package tomcat8.noarch 0:8.5.29-1.77.amzn1 will be updated ---> Package tomcat8.noarch 0:8.5.42-1.80.amzn1 will be an update ---> Package tomcat8-el-3.0-api.noarch 0:8.5.29-1.77.amzn1 will be updated ---> Package tomcat8-el-3.0-api.noarch 0:8.5.42-1.80.amzn1 will be an update ---> Package tomcat8-jsp-2.3-api.noarch 0:8.5.29-1.77.amzn1 will be updated ---> Package tomcat8-jsp-2.3-api.noarch 0:8.5.42-1.80.amzn1 will be an update ---> Package tomcat8-lib.noarch 0:8.5.29-1.77.amzn1 will be updated ---> Package tomcat8-lib.noarch 0:8.5.42-1.80.amzn1 will be an update ---> Package tomcat8-servlet-3.1-api.noarch 0:8.5.29-1.77.amzn1 will be updated ---> Package tomcat8-servlet-3.1-api.noarch 0:8.5.42-1.80.amzn1 will be an update ---> Package tomcat8-webapps.noarch 0:8.5.29-1.77.amzn1 will be updated ---> Package tomcat8-webapps.noarch 0:8.5.42-1.80.amzn1 will be an update --> Finished Dependency Resolution Dependencies Resolved ============================================================================================================================================================================================================================================= Package Arch Version Repository Size ============================================================================================================================================================================================================================================= Updating: tomcat8 noarch 8.5.42-1.80.amzn1 amzn-updates 102 k tomcat8-el-3.0-api noarch 8.5.42-1.80.amzn1 amzn-updates 110 k tomcat8-jsp-2.3-api noarch 8.5.42-1.80.amzn1 amzn-updates 68 k tomcat8-lib noarch 8.5.42-1.80.amzn1 amzn-updates 4.3 M tomcat8-servlet-3.1-api noarch 8.5.42-1.80.amzn1 amzn-updates 255 k tomcat8-webapps noarch 8.5.42-1.80.amzn1 amzn-updates 358 k Transaction Summary ============================================================================================================================================================================================================================================= Upgrade 6 Packages Total download size: 5.2 M Downloading packages: (1/6): tomcat8-el-3.0-api-8.5.42-1.80.amzn1.noarch.rpm | 110 kB 00:00:00 (2/6): tomcat8-8.5.42-1.80.amzn1.noarch.rpm | 102 kB 00:00:00 (3/6): tomcat8-jsp-2.3-api-8.5.42-1.80.amzn1.noarch.rpm | 68 kB 00:00:00 (4/6): tomcat8-webapps-8.5.42-1.80.amzn1.noarch.rpm | 358 kB 00:00:00 (5/6): tomcat8-lib-8.5.42-1.80.amzn1.noarch.rpm | 4.3 MB 00:00:00 (6/6): tomcat8-servlet-3.1-api-8.5.42-1.80.amzn1.noarch.rpm | 255 kB 00:00:00 --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- Total 6.0 MB/s | 5.2 MB 00:00:00 Running transaction check Running transaction test Transaction test succeeded Running transaction Updating : tomcat8-servlet-3.1-api-8.5.42-1.80.amzn1.noarch 1/12 Updating : tomcat8-el-3.0-api-8.5.42-1.80.amzn1.noarch 2/12 Updating : tomcat8-jsp-2.3-api-8.5.42-1.80.amzn1.noarch 3/12 Updating : tomcat8-lib-8.5.42-1.80.amzn1.noarch 4/12 Updating : tomcat8-8.5.42-1.80.amzn1.noarch 5/12 Updating : tomcat8-webapps-8.5.42-1.80.amzn1.noarch 6/12 Cleanup : tomcat8-webapps-8.5.29-1.77.amzn1.noarch 7/12 Cleanup : tomcat8-8.5.29-1.77.amzn1.noarch 8/12 Cleanup : tomcat8-lib-8.5.29-1.77.amzn1.noarch 9/12 Cleanup : tomcat8-jsp-2.3-api-8.5.29-1.77.amzn1.noarch 10/12 Cleanup : tomcat8-el-3.0-api-8.5.29-1.77.amzn1.noarch 11/12 Cleanup : tomcat8-servlet-3.1-api-8.5.29-1.77.amzn1.noarch 12/12 Verifying : tomcat8-webapps-8.5.42-1.80.amzn1.noarch 1/12 Verifying : tomcat8-jsp-2.3-api-8.5.42-1.80.amzn1.noarch 2/12 Verifying : tomcat8-lib-8.5.42-1.80.amzn1.noarch 3/12 Verifying : tomcat8-el-3.0-api-8.5.42-1.80.amzn1.noarch 4/12 Verifying : tomcat8-servlet-3.1-api-8.5.42-1.80.amzn1.noarch 5/12 Verifying : tomcat8-8.5.42-1.80.amzn1.noarch 6/12 Verifying : tomcat8-webapps-8.5.29-1.77.amzn1.noarch 7/12 Verifying : tomcat8-lib-8.5.29-1.77.amzn1.noarch 8/12 Verifying : tomcat8-jsp-2.3-api-8.5.29-1.77.amzn1.noarch 9/12 Verifying : tomcat8-8.5.29-1.77.amzn1.noarch 10/12 Verifying : tomcat8-servlet-3.1-api-8.5.29-1.77.amzn1.noarch 11/12 Verifying : tomcat8-el-3.0-api-8.5.29-1.77.amzn1.noarch 12/12 Updated: tomcat8.noarch 0:8.5.42-1.80.amzn1 tomcat8-el-3.0-api.noarch 0:8.5.42-1.80.amzn1 tomcat8-jsp-2.3-api.noarch 0:8.5.42-1.80.amzn1 tomcat8-lib.noarch 0:8.5.42-1.80.amzn1 tomcat8-servlet-3.1-api.noarch 0:8.5.42-1.80.amzn1 tomcat8-webapps.noarch 0:8.5.42-1.80.amzn1 Complete! [hadoop@ip-10-0-1-84 /]$ yum info tomcat8 Loaded plugins: priorities, update-motd, upgrade-helper 6 packages excluded due to repository priority protections Installed Packages Name : tomcat8 Arch : noarch Version : 8.5.42 Release : 1.80.amzn1 Size : 326 k Repo : installed From repo : amzn-updates Summary : Apache Servlet/JSP Engine, RI for Servlet 3.1/JSP 2.3 API URL : http://tomcat.apache.org/ License : ASL 2.0 Description : Tomcat is the servlet container that is used in the official Reference : Implementation for the Java Servlet and JavaServer Pages technologies. : The Java Servlet and JavaServer Pages specifications are developed by : Sun under the Java Community Process. : : Tomcat is developed in an open and participatory environment and : released under the Apache Software License version 2.0. Tomcat is intended : to be a collaboration of the best-of-breed developers from around the world. |
So it was able to update the Tomcat even though the same command run through bootstrap didn’t do it. What’s happening?
Let’s try installing it manually. Add another boostrap action before the AWS script:
1 2 3 4 5 6 7 8 9 10 11 |
#!/bin/sh set -ex { yum info tomcat8 yum --showduplicates --releasever=latest list tomcat8 sudo yum update --releasever=latest --cve CVE-2019-0232 sudo yum update -y --releasever=latest tomcat8 sudo yum update -y --releasever=latest tomcat8-8.5.42-1.80.amzn1 } >> /home/hadoop/cves.out 2>> /home/hadoop/cves.err |
Logs show this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 |
Loaded plugins: priorities, update-motd, upgrade-helper 6 packages excluded due to repository priority protections Available Packages Name : tomcat8 Arch : noarch Version : 8.5.29 Release : 1.77.amzn1 Size : 102 k Repo : amzn-updates/2017.09 Summary : Apache Servlet/JSP Engine, RI for Servlet 3.1/JSP 2.3 API URL : http://tomcat.apache.org/ License : ASL 2.0 Description : Tomcat is the servlet container that is used in the official : Reference Implementation for the Java Servlet and JavaServer Pages : technologies. The Java Servlet and JavaServer Pages specifications : are developed by Sun under the Java Community Process. : : Tomcat is developed in an open and participatory environment and : released under the Apache Software License version 2.0. Tomcat is : intended to be a collaboration of the best-of-breed developers : from around the world. Loaded plugins: priorities, update-motd, upgrade-helper 8 packages excluded due to repository priority protections Available Packages tomcat8.noarch 8.5.29-1.77.amzn1 amzn-main tomcat8.noarch 8.5.32-1.78.amzn1 amzn-updates tomcat8.noarch 8.5.40-1.79.amzn1 amzn-updates tomcat8.noarch 8.5.42-1.80.amzn1 amzn-updates Loaded plugins: priorities, update-motd, upgrade-helper 8 packages excluded due to repository priority protections No packages needed for security; 126 packages available Resolving Dependencies Loaded plugins: priorities, update-motd, upgrade-helper 8 packages excluded due to repository priority protections Package(s) tomcat8 available, but not installed. No packages marked for update Loaded plugins: priorities, update-motd, upgrade-helper 8 packages excluded due to repository priority protections Package(s) tomcat8-8.5.42-1.80.amzn1 available, but not installed. No packages marked for update |
So it could find version 8.5.42 but refused to update it. How to fix that? The solution is to install instead of update. Let’s try this:
1 2 3 4 5 6 7 8 9 10 11 12 13 |
#!/bin/sh set -ex { sudo yum install -y --releasever=latest tomcat8-servlet-3.1-api-8.5.42 sudo yum install -y --releasever=latest tomcat8-el-3.0-api-8.5.42 sudo yum install -y --releasever=latest tomcat8-jsp-2.3-api-8.5.42 sudo yum install -y --releasever=latest tomcat8-lib-8.5.42 sudo yum install -y --releasever=latest tomcat8-8.5.42 sudo yum install -y --releasever=latest tomcat8-webapps-8.5.42 exit 0 } >> /home/hadoop/cves.out 2>> /home/hadoop/cves.err |
And now it works as expected. Unfortunately, installing specific Tomcat version (instead of latest one) is not the best idea but at least this solves the issue.