Frontier squid release notes ============================ frontier-squid-5.9-2 - 11 January 2024 (carl.vuosalo@cern.ch) - Apply security patches from Squid 6 to address security concerns since a version of Squid 6 suitable for frontier-squid is not available yet. - The following security vulnerabilities have been addressed with patches: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-46724 https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-46847 https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-46848 https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-49285 https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-50269 https://github.com/squid-cache/squid/security/advisories/GHSA-j83v-w3p4-5cqh - The following two vulnerabilities are addressed by disabling Gopher and TRACE requests, respectively, in the squid.conf.proto file: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2023-46728 https://megamansec.github.io/Squid-Security-Audit/trace-uaf.html frontier-squid-5.9-1 - 19 May 2023 (carl.vuosalo@cern.ch) - Update to squid-5.9, with announcement at http://lists.squid-cache.org/pipermail/squid-announce/2023-May/000147.html and release notes at http://www.squid-cache.org/Versions/v5/squid-5.9-RELEASENOTES.html. squid-5.9 contains small fixes including improvement of debug logging related to the reply_body_max_size parameter. - Consistent with squid5, disallow the combination of multiple workers, ufs cache, and memory_cache_shared even if collapsed_forwarding is off. - Limit the maximum number of file descriptors to 65536 even if the OS would allow a higher number. frontier-squid-5.8-2 - 28 March 2023 (carl.vuosalo@cern.ch) - Minor fixes to support proper package creation on EL9. frontier-squid-5.8-1 - 16 March 2023 (carl.vuosalo@cern.ch) - Update to squid-5.8 with release notes at https://www.squid-cache.org/Versions/v5/squid-5.8-RELEASENOTES.html and announcement at http://lists.squid-cache.org/pipermail/squid-announce/2023-February/000145.html Most important new features in 5.8: - A predefined ACL named "to_linklocal" which matches traffic attempting to access link-local network services has been added. It is set to "deny" in squid/files/postinstall/squid.conf.proto. - Bug fixed where cache manager API erroneously returns "mgr_index" instead of requested data. - Add object-*.cloud to the computecanada portion of MAJOR_CVMFS in squid/files/postinstall/squid.conf.proto. frontier-squid-5.7-2 - 31 January 2023 (carl.vuosalo@cern.ch) - Fix bug where old caches were not always cleaned up. frontier-squid-5.7-1 - 02 December 2022 (carl.vuosalo@cern.ch) - Update to squid-5.7 with release notes at http://wiki.squid-cache.org/Releases/Squid-5 and http://www.squid-cache.org/Versions/v5/RELEASENOTES.html. Most important new feature in 5.7: - "Happy Eyeballs" feature uses the first destination IP address that responds from DNS, whether it is IPv4 (A records) or IPv6 (AAAA records). A consequence of this feature is that the dns_v4_first directive is no longer supported. - Add sites cvmfs-stratum-one.cc.kek.jp and cvmfs*.sdcc.bnl.gov to MAJOR_CVMFS in squid/files/postinstall/squid.conf.proto. - Remove obsolete frontier*.racf.bnl.gov from ATLAS_FRONTIER in squid/files/postinstall/squid.conf.proto. frontier-squid-4.17-2 - 20 September 2022 (carl.vuosalo@cern.ch) - Add sites sampacs*.if.usp.br and cvmfs-*.hpc.swin.edu.au to MAJOR_CVMFS in squid/files/postinstall/squid.conf.proto. frontier-squid-4.17-1 - 11 October 2021 (edita.kizinevic@cern.ch) - Update to squid-4.17. Includes squid-4.16 with release announcement at http://lists.squid-cache.org/pipermail/squid-announce/2021-July/000134.html and squid-4.17 with release announcement at http://lists.squid-cache.org/pipermail/squid-announce/2021-October/000137.html The latter includes a security fix, but it is to code disabled in frontier-squid. frontier-squid-4.15-2 - 4 June 2021 (dwd@fnal.gov) - Fix SQUID_COMPRESS_LOGS=false, which was broken since 4.13-3. Change it to also use the external logrotate command instead of the squid builtin log rotation. frontier-squid-4.15-1 - 10 May 2021 (dwd@fnal.gov) - Update to squid-4.15 with release announcement at http://lists.squid-cache.org/pipermail/squid-announce/2021-May/000127.html The update includes security fixes, and at least one (SQUID-2020:11) is relevant because it enables someone that is allowed by access controls to use squid for some purpose to bypass any other access controls and use it for any purpose. That one can be worked around with "uri_whitespace deny", but other denial of service vulnerabilities fixed in this version have no workaround. - Set "uri_whitespace deny" by default to avoid any other related vulnerabilities in the future. - Exit with an error if either of the two required logfile_rotate entries are not in squid.conf. This is for the protection of who are using SQUID_CUSTOMIZE=false and have not properly merged the changes from squid.conf.frontierdefault. - Update the customhelps.awk function setoption() to work correctly after a setoptionparameter(). - Allow cache_dir null type which means no disk cache, for systems that have very large cache_mem and maximum_object_size_in_memory. frontier-squid-4.13-5 - 29 March 2021 (dwd@fnal.gov) - Use /bin/bash instead of $SHELL in fn-local-squid.sh because $SHELL is sometimes set to /bin/nologin. This affected cleaning out old cache files and starting shoal. - Do not attempt to set net.core.somaxconn higher than the kernel maximum of 65335. frontier-squid-4.13-4 - 19 March 2021 (dwd@fnal.gov) - Changes for supporting shoal version 1.0.0 in the rpm - Add Compute Canada stratum 1s to MAJOR_CVMFS - Add any parameters passed in by $SQUID_START_ARGS to the starting of squid. This is set to --foreground by the OSG frontier-squid docker container. frontier-squid-4.13-3 - 17 March 2021 (dwd@fnal.gov) - Change log rotates to no longer use reconfigure signal but to instead set logfile_rotate to zero and use the rotate signal. This is much lighter weight and works around squid bug #5113. Administrators can still set logfile_rotate to non-default values, but it is used only by fn-local-squid.sh and overridden for squid. - With multiple squid services, send the rotate signal only once to each squid. - Add timestamps to squidcron.log for the beginning and end of rotation. frontier-squid-4.13-2 - 28 September 2020 (dwd@fnal.gov) - Move temporary logrotate config file to the log directory, to keep SELinux on CentOS8 happy. frontier-squid-4.13-1 - 24 August 2020 (dwd@fnal.gov) - Update to squid-4.13 with release announcement at https://www.mail-archive.com/squid-announce@lists.squid-cache.org/msg00117.html It includes a couple of relevant security advisories related to cache poisoning. - Remove patch for bug 5051 since it is included in the 4.13 release. frontier-squid-4.12-2 - 29 June 2020 (dwd@fnal.gov) - Note that an additional, more relevant squid advisory fixed in 4.12 was published late: https://www.mail-archive.com/squid-announce@lists.squid-cache.org/msg00116.html - Support the "stdio:" prefix for access_log frontier-squid-4.12-1 - 23 June 2020 (dwd@fnal.gov) - Update to squid-4.12 with release announcement at https://www.mail-archive.com/squid-announce@lists.squid-cache.org/msg00114.html It includes a couple of security advisories, but they are on features that as far as I know are not used by frontier-squid users. - Remove patches for bug #5030 and #5041 which are included in 4.12 - Slightly update patch for bug #5051 as recommended by squid consultant frontier-squid-4.11-4 - 9 June 2020 (dwd@fnal.gov) - Apply patch for bug #5041 which broke compilation in squid-4.11 on el7 with systemd. frontier-squid-4.11-3 - 3 June 2020 (dwd@fnal.gov) - Apply patch for bug #5051 which prevents a negative cache from persisting indefinitely with if-modified-since and collapsed forwarding. - Fix shoal when there are multiple squid workers. - Add cc.*\.in2p3\.fr to MAJOR_CVMFS. It was already in ATLAS_FRONTIER so it was included for installations that accept both, but not for those that accept only MAJOR_CVMFS. frontier-squid-4.11-2 - 23 April 2020 (dwd@fnal.gov) - The 4.11 release announcement is now at https://www.mail-archive.com/squid-announce@lists.squid-cache.org/msg00108.html The announcement was delayed because security vulnerabilities were being double-checked. - frontier-squid was not susceptible to the ESI vulnerability, because of missing libraries on the build machine. Add --disable-esi to make sure it doesn't get accidentally enabled. frontier-squid-4.11-1 - 21 April 2020 (dwd@fnal.gov) - Update to squid-4.11. There's no announcement yet but the change log is at http://www.squid-cache.org/Versions/v4/changesets/. - Remove patches for bugs #5022 and #5036 which are in version 4.11. The patch for #5030 is still applied. frontier-squid-4.10-4 - 13 April 2020 (dwd@fnal.gov) - Add patch for bug #5036 which caused a varying number of capital 'L's to appear at the beginning of access.log lines when the log buffer overflows. frontier-squid-4.10-3 - 16 March 2020 (dwd@fnal.gov) - Update patch for squid bug #5022 to final version. - Add patch for squid bug #5030 which reported that negative caching was not working. This is an important feature for keeping load down on CVMFS stratum 1s that are not hosting a repository. frontier-squid-4.10-2 - 13 February 2020 (dwd@fnal.gov) - Apply patch for squid bug #5022, to prevent a reconfigure from crashing the coordinator process when there are multiple workers. - Use reconfigure signal for log rotation in all cases again. frontier-squid-4.10-1 - 3 February 2020 (dwd@fnal.gov) - Update to squid-4.10, with release notes at https://www.mail-archive.com/squid-announce@lists.squid-cache.org/msg00103.html including a fix for a serious security vulnerability affecting reverse proxies and a potential information leak when proxying ftp. - Remove patch for bug #4735, as it has been included in the 4.10 release. - Change compressing log rotation back to use copytruncate when there are multiple workers, because the reconfigure signal triggers an SNMP bug when there are multiple workers (squid bug #5022). - Disable log rotatation when SQUID_MAX_ACCESS_LOG=0. Using this is highly discouraged since standard logrotate is usually unable to keep up with the high volume of logs typically generated by squid. It also interferes with frontier-awstats support. - Change log rotating cron scripts to ignore commented lines in /etc/sysconfig/frontier-squid frontier-squid-4.9-5 - 27 January 2020 (dwd@fnal.gov) - Change compressing log rotation to not use logrotate's copytruncate. Instead, use create and a postrotate that sends squid a reconfigure signal. This is needed when using rsyslog's imfile module to copy to syslog, but also greatly speeds up rotation when log files are large. frontier-squid-4.9-4 - 31 December 2019 (dwd@fnal.gov) - Limit the hard nofile ulimit check to the "start" operation. frontier-squid-4.9-3 - 31 December 2019 (dwd@fnal.gov) - Apply a patch for bug #4735, to prevent caching an object when its http/1.1 last chunk (that is, zero-length chunk) is missing. - Disable the URN proto by default, to avoid potential future security vulnerabilities. - Fail if the hard nofile ulimit for the squid user is less than 4096 (which is unlikely since it is the default value on RHEL6 & 7) and increase the soft nofile ulimit to match the hard limit. Squid uses the soft limit as the default max_filedescriptors. Previously the root portion of the startup script attempted to increase the soft limit to 4096, but it turned out to have no effect because runuser resets it to its default for that user. frontier-squid-4.9-2 - 10 November 2019 (dwd@fnal.gov) - Fix bug in daily cron that prevented it from rotating any logs frontier-squid-4.9-1 - 8 November 2019 (dwd@fnal.gov) - Update to squid-4.9 with release notes at https://www.mail-archive.com/squid-announce@lists.squid-cache.org/msg00100.html - Add cvmfs01.nikhef.nl, cvmfs-stratum-one.zeuthen.desy.de, and grid-cvmfs-one.desy.de to the MAJOR_CVMFS acl. frontier-squid-4.8-2 - 29 August 2019 (dwd@fnal.gov) - Add support for starting and stopping shoal-agent when installed and enabled by SQUID_AUTO_DISCOVER=true frontier-squid-4.8-1 - 17 July 2019 (dwd@fnal.gov) - Update to squid-4.8 with release notes at https://www.mail-archive.com/squid-announce@lists.squid-cache.org/msg00096.html There was no 4.7 announcement but here's the ChangeLog: https://github.com/squid-cache/squid/blob/f977bfa698ab92e9474c775cef0e01fb756a4b0f/ChangeLog frontier-squid-4.6-2 - 17 May 2019 (dwd@fnal.gov) - Add support for sending access_log through syslog - Add additional allowed monitoring host CERN IP range of 188.185.48.0/20 frontier-squid-4.6-1 - 8 Apr 2019 (dwd@fnal.gov) - Update to squid-4.6 with release notes at https://www.mail-archive.com/squid-announce@lists.squid-cache.org/msg00090.html squid-4.5 release notes are at https://www.mail-archive.com/squid-announce@lists.squid-cache.org/msg00089.html - If a rotate is blocked because processing awstats logs is running long, do the rotate during the next rotateiflarge cron instead. frontier-squid-4.4-1 - 31 Oct 2018 (dwd@fnal.gov) - Update to squid-4.4 with release notes at https://www.mail-archive.com/squid-announce@lists.squid-cache.org/msg00086.html including one fairly significant security fix for a potential denial of service due to memory leaks on rejected SNMP queries. - If different squid services have different numbers of workers (for example by using setserviceoption on "workers") then use the biggest number when creating cache and log directories. frontier-squid-4.3-1 - 4 Oct 2018 (dwd@fnal.gov) - Major update to squid-4.3. - Removed patch for bug 4616 (patch was already applied in 4.3). - Replaced patch for bug 3952 with new --foreground option. - Ported patch for bug 7 forward. frontier-squid-3.5.28-2 - 28 Aug 2018 (dwd@fnal.gov) - Set umask and PATH in fn-local-squid.sh because /etc/init.d/functions used to set them. This fixes problems with log rotation from cron. - Invoke /etc/init.d/functions from /etc/init.d/frontier-squid if it exists, because that is how systemd-based systems invoke systemctl to keep track of the service. If it does not exist, set the PATH. frontier-squid-3.5.28-1 - 23 Aug 2018 (dwd@fnal.gov) - Update to squid-3.5.28. Here is the release announcement: https://www.mail-archive.com/squid-announce@lists.squid-cache.org/msg00083.html Removed patches for advisories SQUID-2018_1 and SQUID-2018_2 and for bug 4767 because fixes are included in the release. - Remove including /etc/init.d/functions from fn-local-squid.sh and from /etc/init.d/frontier-squid. It wasn't being used and isn't in some docker containers such as slc6-lite. - If there are *.log.0.gz files in the presence of awstats, remove them before rotating. They prevented rotation from working on el7. frontier-squid-3.5.27-5 - 6 Jul 2018 (dwd@fnal.gov) - Avoid a reverse DNS lookup for every client connection by using the workaround in https://bugs.squid-cache.org/show_bug.cgi?id=4575, which is to override the defaults for url_rewrite_extras and store_id_extras. frontier-squid-3.5.27-4 - 1 Mar 2018 (dwd@fnal.gov) - If the maximum listen backlog (sysctl net.core.somaxconn) is less than 1/4th the file descriptor limit (ulimit -n) when starting squid, increase it to that amount. That's the listen backlog that squid requests, and the larger backlog helps when a server is hit so hard with requests that squid can't keep up. frontier-squid-3.5.27-3 - 23 Jan 2018 (dwd@fnal.gov) - Add configuration options to always honor "Pragma: no-cache" from a client, even if the client also sends a "Cache-control" header, as the current frontier-client (v2.8.20) always does. https://bugs.squid-cache.org/show_bug.cgi?id=4809 - Included provided patches from the squid project related to two denial of service security advisories which they say could affect all reverse proxies. http://www.squid-cache.org/Advisories/SQUID-2018_1.txt http://www.squid-cache.org/Advisories/SQUID-2018_2.txt For example frontier server launchpad squids are configured as reverse proxies. frontier-squid-3.5.27-2 - 8 Dec 2017 (dwd@fnal.gov) - Add openhtc.io aliases to CMS_FRONTIER, ATLAS_FRONTIER, and MAJOR_CVMFS acls. frontier-squid-3.5.27-1 - 1 Sep 2017 (dwd@fnal.gov) - Upgrade to squid-3.5.27. The release announcement hasn't been published yet but here is the ChangeLog: https://github.com/squid-cache/squid/blob/v3.5/ChangeLog Release notes from prior releases since last frontier-squid release: https://www.spinics.net/lists/squid/msg86349.html (3.5.26) https://www.spinics.net/lists/squid/msg85634.html (3.5.25) - Remove the patch from #2833 because it was updated and included in the 3.5.27 release. - Add a patch from #4767 for making IPv6 SNMP queries work with multiple worker processes instead of causing a crash & restart. - Add IHEP and UNL stratum ones to the MAJOR_CVMFS access control list. frontier-squid-3.5.24-3 - 22 Mar 2017 (dwd@fnal.gov) - Change quick_abort_min and quick_abort_max once again, this time to 0 KB, because someone doing a different repeatable test found that the 1 GB value still caused some crashes but 0 KB didn't. frontier-squid-3.5.24-2 - 17 Mar 2017 (dwd@fnal.gov) - Change the default setting of quick_abort_min from -1 KB to 1 GB and the default setting of quick_abort_max from 16 KB to 1 GB. This is to work around the squid crash documented in squid bug #4554. A consequence is that when a client aborts an object download, the object won't get cached even though it is highly likely that another client will soon want the same object. frontier-squid-3.5.24-1 - 31 Jan 2017 (dwd@fnal.gov) - Upgrade to squid-3.5.24 with release notes at https://www.spinics.net/lists/squid/msg85024.html The bug fixes we had been including are not yet in this release, so all the same patches are kept. The most notable change affecting WLCG in this release was a fix for a bug that caused 'cache deny' to not have any effect. That is used for some cvmfs stratum 1s, and perhaps other applications. frontier-squid-3.5.23-6 - 26 Jan 2017 (dwd@fnal.gov) - Replace the bug #2833 fix with a newer version from the consultant. The full name is SQUID-252-collapsed-slaves-non-sharable-responses-3.5-t10. This one might be final. - Add patch from squid bug #4616, since two users had reported the "mem" assertion failure in store_client.cc. The full name of the patch is bug4616-cf-mem-assert-t1. frontier-squid-3.5.23-5 - 25 Jan 2017 (dwd@fnal.gov) - Replace the bug #2833 fix with a newer version from the consultant. The full name is SQUID-252-collapsed-slaves-non-sharable-responses-3.5-t9. The previous version is suspected in a crash at one grid site. This version required a couple of small changes to apply and compile cleanly. frontier-squid-3.5.23-4 - 13 Jan 2017 (dwd@fnal.gov) - Replace the bug #2833 fix with a newer version from the consultant. The full name is SQUID-252-collapsed-slaves-non-sharable-responses-3.5-t6. It is probably not the final patch. - Add creation of /var/run/squid in the init.d script, if it did not exist. This is because systemd-based systems (EL7) do not preserve the directory across boots, and on some systems the systemd configuration file to create it at boot time does not work. That can happen if the system gets user ids from the network, so the squid user id might not yet be available when the systemd configuration file is used. frontier-squid-3.5.23-3 - 20 Dec 2016 (dwd@fnal.gov) - Replace my patch that comments out the fix for security advisory SQUID-2016:10 with a patch from the consultant for bug #2833. This patch makes collapsed forwarding work again with If-Modified-Since while leaving the security advisory fixed. frontier-squid-3.5.23-2 - 20 Dec 2016 (dwd@fnal.gov) - Comment out the fix for security advisory SQUID-2016:10, because it caused collapsed forwarding to break for If-Modified-Since requests. It was probably implemented incorrectly; discussion is in the reopened squid bug report #2833. frontier-squid-3.5.23-1 - 19 Dec 2016 (dwd@fnal.gov) - Upgrade to squid-3.5.23 with release notes at http://www.spinics.net/lists/squid/msg84536.html The most notable changes are fixes for two privacy vulnerabilities. They don't affect the applications primarily used with frontier-squid but may affect other applications. - Add the CERN IPv6 monitoring address range to the HOST_MONITOR acl. - Add IPv6 private net addresses to default setting of NET_LOCAL. frontier-squid-3.5.22-2 - 14 Oct 2016 (dwd@fnal.gov) - Change default value of dns_v4_first to "on", so it will always try ipv4 first if available and then ipv6. The default was "off" which always tries ipv6 first. frontier-squid-3.5.22-1 - 13 Oct 2016 (dwd@fnal.gov) - Upgrade to squid-3.5.22 with release notes at http://www.spinics.net/lists/squid/msg83740.html - Remove patches from Alex Rousskov because they are now in the official squid 3.5 release. frontier-squid-3.5.21-2 - 13 Sep 2016 (dwd@fnal.gov) - Fix error in the new logformat. It should be "%>Hs", not ">%Hs". frontier-squid-3.5.21-1 - 13 Sep 2016 (dwd@fnal.gov) - Upgrade to squid-3.5.21 with release notes at http://www.spinics.net/lists/squid/msg83169.html - Remove patch for bug #4428, it is now in the release. - Apply patch from Alex Rousskov for squid bug #4471, fixing collapsed forwarding revalidation when there is no If-Modified-Since. - Change "%Hs" in the default log format to the new style ">%Hs" for squid-3. - Change the default cache_dir size in squid.conf.proto to 10000 MB in case someone deletes the default 10000 MB line in customize.sh. - Change the script that applies patches to exit with an error if any of the patches are not completely applied. frontier-squid-3.5.20-3 - 22 Aug 2016 (dwd@fnal.gov) - Apply patch from the squid team for bug #4428. This bug causes the Cache-control: stale-if-error header, which Frontier uses, to be malformed in cached objects. - Fix bug that caused access logs shared by multiple workers to be rotated multiple times, once for each worker, instead of just once. - When using compressed logs and SQUID_CLEAN_CACHE_ON_START is true (both of which are default), then truncate the swap.state file in ufs cache directories each time logs are rotated. Otherwise the file grows without bounds. - When using the 'restart' function, clean ufs cache directories the same way as when doing 'start'. frontier-squid-3.5.20-2 - 8 Aug 2016 (dwd@fnal.gov) - The infamous squid bug #7 had partially cropped up again. The Date headers on cached 304 Not Modified responses on large objects was returning the original Date rather than a new re-validated Date, even though the Age header was correct. This causes many duplicated queries to get sent upstream. This release works around the problem as discussed here: http://bugs.squid-cache.org/show_bug.cgi?id=7#c79 frontier-squid-3.5.20-1 - 21 Jul 2016 (dwd@fnal.gov) - Upgrade to squid-3.5.20. This is the release announcement: http://www.spinics.net/lists/squid/msg82165.html - Replace my own patches for squid bugs #4311 and #4471 with a big patch from Alex Rousskov fixing collapsed revalidation. It addresses bug #4311 and more importantly bug #2833, where collapsed forwarding did not work with If-Modified-Since on expired objects. Unfortunately it breaks the functionality formerly fixed in the patch for bug #4471; that is, collapsed forwarding now does not work on expired objects that do *not* have a Last-Modified header. This should be no problem for the Frontier application because any response without Last-Modified is given a year expiration time. However, this is a potential issue for CVMFS because the stratum 1s currently expire most objects after 3 days. At least the issue will be relatively infrequent, and if a fix for this bug does not come soon we can probably eventually get the stratum 1 configurations changed to increase the expiration time. frontier-squid-3.5.19-3 - 16 Jun 2016 (dwd@fnal.gov) - Change default minimum_expiry_time to 0 seconds. Without this change, squid will not cache any objects that are loaded for the first time during the last minute of their lifetime; instead, all requests for that object during that minute are sent upstream. It only affects squids that are fed by other squids. frontier-squid-3.5.19-2 - 10 Jun 2016 (dwd@fnal.gov) - Add support for SQUID_CUSTOMIZE environment variable. If set to false it skips generating squid.conf from squid.conf.frontierdefault and customize.sh. Instead, the user is on his or her own to manage squid.conf. It is recommended to start from squid.conf.default because that is less likely to change between releases. The default setting of SQUID_CUSTOMIZE is true. frontier-squid-3.5.19-1 - 11 May 2016 (dwd@fnal.gov) - Upgrade to squid-3.5.19. This is the release announcement: http://www.spinics.net/lists/squid/msg81196.html - Add hepvm.cern.ch to the MAJOR_CVMFS acl frontier-squid-3.5.15-2 - 28 Mar 2016 (dwd@fnal.gov) - Disable memory_cache_shared by default, and make it an error if somebody tries to turn it on without rock cache. - Replace patch for bug #4312 with the one for #4311. It has the same effect and is simpler. frontier-squid-3.5.15-1 - 23 Mar 2016 (dwd@fnal.gov) - Upgrade to squid-3.5.15. This is the release announcement: http://www.spinics.net/lists/squid/msg79992.html - Apply patch for squid bug #2831, to send Cache-control header through on 304 Not Modified responses. - Apply patch for squid bug #4471, to make collapsed forwarding work when there's a previously cached but expired object. It still does not work with If-Modified-Since 304 Not Modified responses (bug #2833). So this makes it work with CVMFS but not yet Frontier. frontier-squid-3.5.9-1 - 15 Oct 2015 (dwd@fnal.gov) - Upgrade to squid-3.5.9. This is the release announcement: http://www.spinics.net/lists/squid/msg77242.html - Pass the SQUID variables in /etc/sysconfig/frontier-squid to the daily log rotation cron, in particular so SQUID_NUM_SERVICES will apply. - When installing with the "proto_install" make target, for rpm, install cron scripts in /usr/share/squid/cron instead of /etc/squid/cron. frontier-squid-3.5.7-1 - 16 Sep 2015 (dwd@fnal.gov) - Upgrade to squid-3.5.7. This is the release announcement: http://www.spinics.net/lists/squid/msg76566.html - Extend the "setoption" macro in customize.sh to work with options that are only identified by a "TAG:" comment in squid.conf, because not all options have a commented example like they did in squid2. - Add support for workers > 1, especially including recognizing the SMP macros that split up directories for separate worker processes. Ensure that the user has either enabled this for the cache directory or chosen rock cache type. For details on the SMP macros see "SMP-Related Macros" in squid.conf comments. - Enable support for the "rock" cache_dir store type, for sharing a cache between multiple SMP workers. Note that this cache type is still susceptible to squid bug #7 so it should be considered only experimental at this point. It may be safely used if it is guaranteed that no application will use If-Modified-Since (for example, if it is only used for CVMFS). - Enable support for the "diskd" cache_dir store type, which is just like "ufs" except that it uses a separate process for disk i/o. - Add support for running multiple independent squid services via the environment variable SQUID_NUM_SERVICES. This assigns each squid a "service name" that is a number from 0 to $SQUID_NUM_SERVICES-1. This value can be accessed with the macro ${service_name} which must be included in the cache_dir, access_log, cache_log, and pid_filename options. Add new customize.sh macro "setserviceoption" for setting options with a numerical value (e.g. http_port) to a different value per service. The value may also be a comma separated list of numbers (e.g. cpu_affinity_map). For usage see customhelps.awk. Each service may have multiple workers if desired. - Remove old wrapper script support for multiple processes since squid3 does everything natively in squid.conf. That includes removing support for automatically setting the core affinity on the separate processes. Instead set the option cpu_affinity_map to get the most performance out of each core. Setting core affinity improves performance by about 15%. - Reduce the verbose debug messages that were coming out at startup. - Add a patch for bug http://bugs.squid-cache.org/show_bug.cgi?id=3952 to prevent the initialization of the cache directory with multiple workers from running in the background. - Add a patch for bug http://bugs.squid-cache.org/show_bug.cgi?id=4312 to add a configuration option collapsed_forwarding_shared_entries_limit. This enables controlling the sharing of collapsed forwarding between SMP workers. This sharing causes deadlocks with the default ufs cache type, so this is set to zero in the default squid.conf. It does not cause deadlocks with rock cache, so to enable it use the comment() macro in customize.sh to drop back to the compiled-in default. - Extend the rotate lock to also cover awstats generation, to prevent more than one from running at a time. frontier-squid-3.5.4-1 - 20 May 2015 (dwd@fnal.gov) - Upgrade to squid-3.5.4, including a fix for a security vulnerability, release announcement here: http://www.spinics.net/lists/squid/msg75017.html - Merge changes from frontier-squid 2.7.STABLE9-22-1 and 2.7.STABLE9-23-1, which were: - Back out the dividing up of file descriptors between multiple squids that was added in frontier-squid-2.7.STABLE9-20. It was based on a confusion over how the limit worked; the file descriptor limit works per process and not per user. - Support new configuration option SQUID_MULTI_PEERING=false to not insert cache_peer parent settings when there are multiple squids. By default when there are multiple squids, any squid other than the first one reads from the first one like it always used to. - Support using multiple squids for a reverse proxy. Formerly it clobbered the http_port and cache_peer parent settings when using multiple squids. Now it preserves any extra parameters on http_port and sets SQUID_MULTI_PEERING=false if a cache_peer parent setting already exists. - Support awstats with multiple squids: invoke run_awstats.sh if it exists (installed by frontier-awstats rpm) for the logs of all of the squids and not just the first one. Requires frontier-awstats rpm version 6.9-3.2 or newer to work properly. - Don't invoke awstats if SQUID_SUFFIX is set (that is, in the frontier-squid2 rpm) so it won't get invoked twice when it is installed simultaneously with frontier-squid. - Support a "daemon:" prefix on access_log and cache_log, a poorly documented squid feature that uses a separate process to handle writing to log files so the main squid process doesn't have to wait for disk I/O. This was added because log compression was observed on one machine to interfere with squid I/O accesses. Make this the default for access_log. - Run log rotation with ionice -n7. - Make slight correction to MAJOR_CVMFS acl regular expression. - Expand the server names allowed at RAL in the ATLAS_FRONTIER acl. frontier-squid-3.5.3-1 - 6 April 2015 (dwd@fnal.gov) - Upgrade to squid-3.5.3, release announcement here: http://lists.squid-cache.org/pipermail/squid-announce/2015-March/000014.html frontier-squid-3.5.2-1 - 19 February 2015 (dwd@fnal.gov) - Upgrade to squid-3.5.2, release announcement here: http://lists.squid-cache.org/pipermail/squid-announce/2015-February/000012.html frontier-squid-3.5.1-1 - 1 January 2015 (dwd@fnal.gov) - Upgrade to first full squid3.5 release, 3.5.1, release announcement here: http://lists.squid-cache.org/pipermail/squid-announce/2015-January/000009.html frontier-squid-3.5.0.4-1 - 12 December 2014 (dwd@fnal.gov) - Upgrade to squid beta release 3.5.0.4, release announcement here: http://lists.squid-cache.org/pipermail/squid-announce/2014-December/000007.html - Merge changes from frontier-squid-2.7.STABLE9-21 which were: - Fix redirection of stderr in the hourly and daily crons so error messages properly go to squidcron.log. - Only require the minimum 4096 file descriptors when doing one of the commands that contain "start" (that is, "start", "restart", or "condrestart") with multiple squids. - Add the script that generates squid.conf to the list of files that trigger regenerating squid.conf if they're newer than squid.conf. - Only generate the per-squid configuration files used with multiple squids when squid.conf is newer than them. - Merge change from a pre-release of frontier-squid-2: - Make slight correction to MAJOR_CVMFS acl regular expression frontier-squid-3.5.0.2-1 - 10 November 2014 (dwd@fnal.gov) - Upgrade to squid beta release 3.5.0.2. Made as minimal changes as possible to the frontier-squid packaging for an early first look at a squid-3 with most of the features required by grid applications. frontier-squid-2.7.STABLE9-20 - 6 November 2014 (dwd@fnal.gov) - Increase the maximum number of squids that may be started from 4 to 16 - When running N > 1 squids, limit each squid to the hard limit on file descriptors divided by N. This limiting greatly reduces or eliminates the number of failed accesses to cache files indicated by TCP_SWAPFAIL_MISS entries in access.log. Require a minimum of 4096 file descriptors for each squid, unless customize.sh sets a value below the calculated limit. - Support use of SQUID_SUFFIX to add a suffix to all the files. This is not supported for use in the standalone tarball, just for use within an rpm. frontier-squid-2.7.STABLE9-19 - 15 September 2014 (dwd@fnal.gov) - Make the default SQUID_MAX_ACCESS_LOG be 5G instead of 1G unless log compression is disabled. This should take about the same maximum space (~11Gbytes) as uncompressed log files did with a max size of 1G. - Add the TRIUMF CVMFS stratum 1 to the list in MAJOR_CVMFS. frontier-squid-2.7.STABLE9-18 - 3 September 2014 (dwd@fnal.gov) - Protect rotate operations with a lock, because now that they compress files they can take a long time. This is especially important for the one minute between the daily cron and the first hourly cron. - Fix bug introduced in last release where if access_log is set to "none", the cache log is rotated every 15 minutes. frontier-squid-2.7.STABLE9-17 - 22 August 2014 (dwd@fnal.gov) - Update the CERN Hungary Data Center's LHCOPN IP address range in the HOST_MONITOR access control list from the incorrect 188.185.0.0/17 to the correct 188.184.128.0/17 and 188.185.128.0/17. - Add commented-out acls CMS_FRONTIER, ATLAS_FRONTIER, and MAJOR_CVMFS that can be uncommented and used in place of RESTRICT_DEST to restrict outgoing connections to the corresponding servers. This allows updating the lists via frontier-squid package upgrades rather than requiring individual administrators to know how to keep the lists up to date. - Include the real time zone in the access.log timestamp instead of always +0000, and include milliseconds after the seconds. - Add the "cvmfs-info" header to the same double-quoted log entry that now has "X-Frontier-Id". Since no client sends both headers, only one will show at a time; frontier entries will end with " -" and cvmfs entries will start with "- ". Cvmfs clients currently only send cvmfs-info if configured with CVMFS_SEND_INFO_HEADER=yes so if that's not the case their log entries will show "- -". - Accept SQUID_MAX_ACCESS_LOG as the variable setting the maximum access log file size in place of LARGE_ACCESS_LOG (which is still accepted for backward compatibility). Also if the value ends in 'M' it indicates megabytes and if it ends in 'G' it indicates gigabytes; the default is bytes. - Run the "hourly" cron 4 times an hour, to catch faster when a log file has gone over the max size limit. - Compress log files by default, using logrotate. If environment variable SQUID_COMPRESS_LOGS is exported and set to 'false', fall back to the previous method of telling squid to rotate the log files without compression. In either case the logfile_rotate configuration parameter is used to define the maximum number of rotated files. If frontier-awstats is also installed, the first file is left uncompressed. When switching either way between compressed and uncompressed, removes all log files of the old type. - Rotate cache.log even if the access_log configuration parameter is "none". - Add a new "removecache" option to the init script to simply remove all of the cache, for use when removing the package. frontier-squid-2.7.STABLE9-16 - 10 May 2013 (dwd@fnal.gov) - Rearrange the new code in the /etc/init.d startup script to be easier to modify at post-install time by the rpm - Change default chkconfig levels to "-" to match the Redhat standard - Change the delay after starting squid from 3 to 10 seconds before checking to see if it is running. That was the previous delay, and a system with very slow disk access running two squids didn't get both of them started within 3 seconds while a cache cleaning was happening in the background. frontier-squid-2.7.STABLE9-15 - 8 May 2013 (dwd@fnal.gov) - Put squidcron.log in the same directory as cache.log rather than access.log, in case the access_log option is set to "none". In the previous version it would put squidcron.log in the squid user's home directory if access_log was "none". - Change the init.d startup script to abort with an error message if the squid user's home directory does not exist, because on RHEL6-derived systems if a user's home directory doesn't exist then cron won't run the user's jobs. - Run squid without the -S option so it will never run an audit of the cache files. During a normal start the cache is deleted so it doesn't matter, and the audit operation can take a very long time on a large cache during a restart. Also an analysis showed that the typical types of errors the audit catches (missing files) are survivable. - Allow multiple background cache cleans to be happening at the same time, in case the cache is very large and someone does multiple stop/start operations. - Add environment variable SQUID_CLEAN_CACHE_ON_START which defaults to true and when set to false prevents clearing the cache on start. It can be set and exported in the package's /etc/sysconfig file. frontier-squid-2.7.STABLE9-14 - 29 Mar 2013 (dwd@fnal.gov) - Put the tarball version string into the string reported as the version in the SNMP-based monitoring. Requires keeping RELEASE variable up to date squid/Makefile. - Supply defaults for cache memory size and cache dir size in the configure scripts, if people just hit enter. frontier-squid-2.7.STABLE9-13 - 28 Jan 2013 (dwd@fnal.gov) - Move the output from the cron jobs from daily.log in the cron directory to squidcron.log in the log directory frontier-squid-2.7.STABLE9-12 - 25 Jan 2013 (dwd@fnal.gov) - Change the comment in customize.sh so the rpm can refer to the standard init script instead of the nonstandard fn-local-squid.sh, and also mention doing reload for a running squid. - Create the default cache directory when doing a make install. frontier-squid-2.7.STABLE9-11 - 11 Jan 2013 (dwd@fnal.gov) - Add the Referer and User-Agent headers to the end of the default log format. Each are in double quotes, and if the headers are missing a hyphen is inserted. - Include the upstream squid source in the tarball. Source rpms are expected to be self-contained, and this tarball is included in the frontier-squid rpm so it has to be self-contained too. frontier-squid-2.7.STABLE9-10 - 20 Dec 2012 (dwd@fnal.gov) - Change the default allowed monitoring hosts to be the "WLCG" address ranges at the main CERN data center and the new backup data center in Hungary - Disable the icp port by default frontier-squid-2.7.STABLE9-9 - 19 Sep 2012 (dwd@fnal.gov) - Fix sed command in frontier-squid-utils/Makefile that caused tarball installation to fail. The bug was introduced in version STABLE-6. It does not affect the rpm. frontier-squid-2.7.STABLE9-8 - 4 Sep 2012 (dwd@fnal.gov) - Change the init.d 'rotate' command to first remove the oldest log files, rather than asking squid to do it, because it can take a long time to delete large access logs and squid stops servicing requests during the rotate process. frontier-squid-2.7.STABLE9-7 - 2 Aug 2012 (dwd@fnal.gov) - Fix bug that prevented the hourly cron from reading an /etc/sysconfig file for the setting of LARGE_ACCESS_LOG, making it always rotate at the default 1GB. Now it will use the setting of that variable from /etc/sysconfig/frontier-squid or /etc/sysconfig/frontier-squid.sh. frontier-squid-2.7.STABLE9-6 - 25 Jun 2012 (dwd@fnal.gov) - Support running up to 4 squids listening on the same port. In order to start multiple squids, subdirectories have to be created for the cache directory of the form squidN where N goes from 0 to the number of squids minus 1. Corresponding directories for the logs and pid file are also used and created if necessary. See more details in the comments at the beginning of fn-local-squid.sh. - Whenever SETSQUIDAFFINITY=true, the affinity of squid processes are tied to individual CPU cores with the use of the taskset command. This has been observed to increase throughput on squids that are CPU-limited by about 15%. When $SETSQUIDAFFINITY is not set, the default is to enable it if and only if more than one squid is being started and the number of cores is greater than or equal to the number of squids. Cores are assigned beginning at number 0 and going up 1 at a time. - Fix the detection of whether or not squid is already running on RHEL6-based systems. - If the maximum number of open file descriptors (ulimit -n) is less than 4096 per process when starting squid, the limit will be increasd to that number. A higher number can be set in the /etc/sysconfig file matching the name of the init.d script (that is, frontier-squid for the rpm and frontier-squid.sh for the tarball) or by setting the 'nofile' parameter in /etc/security/limits.conf. - Eliminate error from 'find' in the init.d script that occurred when generating squid.conf from within a working directory that's not accessible to the user id that squid runs under. - Require access_log to be either "none" or to be in the same directory as cache_log's directory. This simplifies especially the handling of running multiple squids and is usual practice anyway. - Do better checking for error conditions in fn-local-squid.sh: make sure it is not run as root; make sure cache_dir is not set to '/' (so the clean-up of the cache can't try to delete everything); make sure that cache_dir, cache_log, and pid_filename are set and not empty; and make sure that cache_dir, the directory of cache_log, and the directory of pid_filename all exist and are writable. - Integrate better with the rpm distribution so it doesn't need to patch any of the .proto files. - Allow SNMP queries from the two individual IP addresses of the central frontier monitoring machines in addition to the shared virtual IP address. frontier-squid-2.7.STABLE9-5 - 29 Nov 2010 (dwd@fnal.gov) - In order to prevent accidental clobbering of a squid.conf from pre-customize.sh squid installations, print an error and refuse to start if squid.conf has write permission. Also when squid.conf is overwritten, print the path to the saved squid.conf.old. - Make detection of an existing running squid process more robust by matching the process ID in the saved squid.pid file with ones found by the "pidof" Linux command. This prevents an old squid.pid from matching some other process using the same pid after a reboot. - When cleaning out an old cache, move the old cache directories out of the way, remove them in the background, and then proceed so squid will start faster. - Remove the hidden default of always allowing incoming access to private network addresses 10.0.0.0/8 172.16.0.0/12 192.168.0.0/16. Instead, make that default explicit in the first customize.sh that is generated so if the administrator doesn't want that it can be easily changed. - Add to the init.d script to source in /etc/sysconfig/frontier-squid.sh if it exists (for the tarball packaging, that is; for the rpm packaging it is /etc/sysconfig/frontier-squid). - Explicitly choose a shell for runuser from inside the init.d script in case the login shell of the user is not bash - Add new 'proto_install' Makefile targets in frontier-squid-utils and squid source directories that make it easier to create an rpm with installation paths that fit well with standard Linux. Also change the .proto files to have independently specifiable installation paths so the same files can be used for both tarball and rpm packages. frontier-squid-2.7.STABLE9-4 - 18 May 2010 (bjb@jhu.edu) - Replacement SNMP monitor IP address for Fermilab (131.225.240.232) frontier-squid-2.7.STABLE9-3 - 5 April 2010 (dwd@fnal.gov) - The cache.log error message Failed SNMP agent query from : 128.142.202.212. was happening every 3 hours, every time the DNS Time-To-Live ran out on cmsdbsfrontier.cern.ch. To work around that, added frontier.cern.ch to the HOST_MONITOR acl, which will work until that IP address changes, and then the above error message will come back until squid restarts. Details are in http://bugs.squid-cache.org/show_bug.cgi?id=2894. - Add "rotate" and "rotateiflarge" commands to fn-local-squid.sh. Change daily.sh to use "rotate", and add an hourly.sh to use "rotateiflarge". Also add a cron entry to the default crontab to call hourly.sh every hour. This will keep access access.log from growing excessively. The size of access.log that will trigger the hourly rotate is a gigabyte by default, but can be overridden by setting the environment variable LARGE_ACCESS_LOG to the number of bytes desired before invoking hourly.sh from cron. Also change daily.sh to clean up the log file written by both daily.sh and hourly.sh once a week. - Change the "status" command in fn-local-squid.sh to be more robust. frontier-squid-2.7.STABLE9-2 - 1 April 2010 (dwd@fnal.gov) - There's now often a new fairly harmless message Failed SNMP agent query from : 128.142.202.212. that appears in cache.log the first time squid is polled by the MRTG server after startup. That was interfering with the logic in fn-local-squid.sh which was checking for whether or not squid started successfully. This can cause the start script to wait for 5 minutes if the message appears in the first 10 seconds after startup. Change the script to allow for the success message to be in any of the last 5 lines. - Note that having update_headers on sometimes results in a small number of other harmless messages to cache.log of the form storeUpdateCopy: Aborted frontier-squid-2.7.STABLE9-1 - 31 March 2010 (dwd@fnal.gov) - Change the name of the package to make it obvious what the underlying squid release is. - Upgrade to squid-2.7.STABLE9. - Apply patches for the following squid bugs. These are needed only for squids that feed other squids: - http://bugs.squid-cache.org/show_bug.cgi?id=2831 - needed for changing expiration time of cached objects. - http://bugs.squid-cache.org/show_bug.cgi?id=2833 - needed for correct collapsed forwarding of simultaneous If-Modified-Since requests on previously cached objects. - Add a new way to customize squid.conf that gets preserved across upgrades. A script called squid/etc/customize.sh is now used to edit squid.conf, and it is guaranteed to not be overwritten after upgrades. The 3 customization questions asked at configure time are now used to generate customize.sh the first time only, and on upgrades the questions no longer need to be asked. The user may add any additional edits as desired, and functions that understand the squid.conf are provided to make it fairly easy to do. See the comments there and in customhelps.awk for details. Questions from configure can be avoided completely if the user passes a new '--prefix' parameter defining where to install, and also the new '--oldprefix' parameter defining where the old installation is if it is different than --prefix. Whenever customize.sh is changed, squid.conf is automatically recreated when fn-local-squid.sh is run. - Make several changes to the function of init.d/frontier-squid.sh: - Change its implementation to use runuser instead of su, and to always pass all parameters to $INSTALL_DIR/utils/bin/fn-local-squid.sh after switching to the non-privileged user id. - Attempting to start squid while squid is running was supposed to abort before deleting the cache, but if there was a parse error in squid.conf there was a bug that made it proceed to delete the cache even while squid was running. That bug is now fixed: any start attempt while there's a squid.conf parse error will abort before deleting the cache. - Change restart command to not delete the cache. - Implement the condrestart command to only attempt to restart if squid is already running. - Change the previously undocumented reload command to tell squid to reconfigure rather than restarting it. - Add a cleancache command. - Make error codes from all the commands be returned to the shell. - Make several changes to the default squid.conf: - Instead of having a CMSFRONTIER acl that was unused by default, add a commented-out RESTRICT_DEST acl that gives an example on how to restrict the destinations to only cmsfrontier.cern.ch servers. This may be customized via customize.sh. Uses dstdom_regex instead of dst because the latter only works on IP addresses that are cached at the time squid starts. - Remove the cmsdbsfrontier.cern.ch from the HOST_MONITOR acl and instead put it in a HOST_MONITOR_NAME acl that is of type srcdomain rather than type src, because src (like dst) doesn't recognize an IP address change without restarting squid. Would have preferred to use the alias frontier.cern.ch, but unfortunately squid has no source acl type that will both recognize changing IP addresses and work with DNS aliases. Details are in http://bugs.squid-cache.org/show_bug.cgi?id=2837. - Added explicit path settings for all options that locate files, to make it easier for the rpm version of this package to be relocatable. - Stop turning the update_headers option off by default. This is needed by squids that feed other squids in order to support If-Modified-Since, and even though currently non-launchpad squids don't feed other squids that will likely change in the future. - Change the default umask to 022 so log files will be readable by other. - Add a 'make distclean' target to clean out all generated files and get the source directory ready for distribution. frontier_squid4.0rc9 - 17 September 2009 (bjb@jhu.edu) - Upgrade to squid-2.7.STABLE7. The only significant change is that it it is a stable release that includes patch2451. frontier_squid4.0rc8 - 13 September 2009 (bjb@jhu.edu) - Include patch2451 to fix a squid performance bug that causes already-cached items at a site to triple the number of transactions. This especially harms sites that are far away from the central server. frontier_squid4.0rc7 - 29 July 2009 (bjb@jhu.edu) - Add an extra safety check in the start script frontier_squid4.0rc6 - 8 February 2009 (bjb@jhu.edu) - Upgrade to squid-2.7.STABLE6 to close a potential Denial Of Service vulnerability. frontier_squid4.0rc5 - 31 January 2009 (bjb@jhu.edu) - Set "update_headers off" by default because it was suspected to be the cause of some problems seen and is only needed to be on for squids that feed other squids. frontier_squid4.0rc4 - 31 October 2008 (bjb@jhu.edu) - Upgrade to squid-2.7.STABLE5 - Update monitor IP address - New default access.log format - New restart command