From f155b0199ef24479956524d80719cfc00fdd1b4b Mon Sep 17 00:00:00 2001 From: AnyOldName3 Date: Mon, 1 Sep 2025 00:15:30 +0100 Subject: [PATCH 01/12] Undo part of !4759 https://gitlab.com/OpenMW/openmw/-/merge_requests/4759#note_2606465930 was marked as resolved without really being resolved, so bypassed normal review and was merged prematurely. We tell developers to run these scripts on their machines, so it's rude to install things we don't have to over the top of an existing installation. If the existing installation was via brew via the default sources, then it's clever enough not to break things, but if it's using custom sources or the install is outside of brew, it'll either break or overwrite what's there. The three tools we've historically checked before overwriting all have an upstream MacOS distribution that isn't through brew, so it's decently likely that they exist, and we've never agreed to drop support for them. I went a little further than the original code: * I added a check before overwriting brew itself. I don't know whether the install script would notice this anyway. * I added a check to ensure it's specifically Qt 6.x that's already installed as we dropped support for Qt 5. * I added the checks back to both scripts even though !4759 only removed them from one. There only used to be one script, and when it was split, the checks only made it to one copy. I could have gone further by adding checks for other tools before we install them, but we didn't have these in the past and I couldn't be bothered. Try ARCHPREFERENCE environment variable Go back to using subshell ARCHPREFERENCE won't retroactively relaunch the current shell. Also use /usr/local/bin/brew as the x86_64 version we just installed isn't added to the path by default. Fresh shell for brew installer? escape things better This commit message was very nearly just emoji or a rant about sh-compatible shells. what is going on? Nested quotes? --- CI/macos/before_install.amd64.sh | 8 ++++++-- CI/macos/before_install.arm64.sh | 4 +++- 2 files changed, 9 insertions(+), 3 deletions(-) diff --git a/CI/macos/before_install.amd64.sh b/CI/macos/before_install.amd64.sh index 5ed4f23b21..1cf403e5ea 100755 --- a/CI/macos/before_install.amd64.sh +++ b/CI/macos/before_install.amd64.sh @@ -1,8 +1,12 @@ #!/bin/sh -ex -arch -x86_64 /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" +command -v /usr/local/bin/brew || arch -x86_64 bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" -arch -x86_64 /usr/local/bin/brew install curl xquartz gd fontconfig freetype harfbuzz brotli ccache cmake qt@6 openal-soft icu4c yaml-cpp sqlite +arch -x86_64 bash -c "command -v ccache || /usr/local/bin/brew install ccache" +arch -x86_64 bash -c "command -v cmake >/dev/null 2>&1 || /usr/local/bin/brew install cmake" +arch -x86_64 bash -c "command -v qmake >/dev/null 2>&1 && qmake -v | grep -F 'Using Qt version 6.' >/dev/null || /usr/local/bin/brew install qt@6" + +arch -x86_64 /usr/local/bin/brew install curl xquartz gd fontconfig freetype harfbuzz brotli openal-soft icu4c yaml-cpp sqlite curl -fSL -R -J https://gitlab.com/OpenMW/openmw-deps/-/raw/main/macos/openmw-deps-20240802.zip -o ~/openmw-deps.zip unzip -o ~/openmw-deps.zip -d /tmp > /dev/null diff --git a/CI/macos/before_install.arm64.sh b/CI/macos/before_install.arm64.sh index d1fb572f01..6939fb3a2d 100755 --- a/CI/macos/before_install.arm64.sh +++ b/CI/macos/before_install.arm64.sh @@ -3,9 +3,11 @@ brew tap --repair brew update --quiet +command -v ccache >/dev/null 2>&1 || brew install ccache command -v cmake >/dev/null 2>&1 || brew install cmake +command -v qmake >/dev/null 2>&1 && qmake -v | grep -F "Using Qt version 6." >/dev/null || brew install qt@6 -brew install curl xquartz gd fontconfig freetype harfbuzz brotli qt@6 ccache openal-soft icu4c yaml-cpp sqlite +brew install curl xquartz gd fontconfig freetype harfbuzz brotli openal-soft icu4c yaml-cpp sqlite curl -fSL -R -J https://gitlab.com/OpenMW/openmw-deps/-/raw/main/macos/openmw-deps-20240818-arm64.tar.xz -o ~/openmw-deps.tar.xz tar xf ~/openmw-deps.tar.xz -C /tmp > /dev/null From 9fe8f3b03c029724f2006cd84e572455d093252b Mon Sep 17 00:00:00 2001 From: AnyOldName3 Date: Mon, 1 Sep 2025 18:46:41 +0100 Subject: [PATCH 02/12] Attempt to use Qt not necessarily from brew The homebrew package should add qmake to the path, and qmake knows where Qt is. If this works for brew, it should work for not-brew, too. This *might* give a trailing /lib on the path CMake sees, but CMake implicitly tries adding a /lib suffix to paths in CMAKE_PREFIX_PATH, so this shouldn't make a difference to anything. --- CI/before_script.macos.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CI/before_script.macos.sh b/CI/before_script.macos.sh index 15be5eb4e0..3ec72cebaa 100755 --- a/CI/before_script.macos.sh +++ b/CI/before_script.macos.sh @@ -10,11 +10,11 @@ cd build DEPENDENCIES_ROOT="/tmp/openmw-deps" if [[ "${MACOS_AMD64}" ]]; then - QT_PATH=$(arch -x86_64 /usr/local/bin/brew --prefix qt@6) + QT_PATH=$(arch -x86_64 /bin/bash -c "qmake -v | sed -rn -e 's/Using Qt version [.0-9]+ in //p'") ICU_PATH=$(arch -x86_64 /usr/local/bin/brew --prefix icu4c) OPENAL_PATH=$(arch -x86_64 /usr/local/bin/brew --prefix openal-soft) else - QT_PATH=$(brew --prefix qt@6) + QT_PATH=$(qmake -v | sed -rn -e "s/Using Qt version [.0-9]+ in //p") ICU_PATH=$(brew --prefix icu4c) OPENAL_PATH=$(brew --prefix openal-soft) fi From 3bf68d44b796c216ef5b72de5f7df29e66398513 Mon Sep 17 00:00:00 2001 From: AnyOldName3 Date: Wed, 3 Sep 2025 19:29:18 +0100 Subject: [PATCH 03/12] Try using native CMake and CCache In principle, CMake should be able to cross-compile just fine and CCache shouldn't care. Maybe homebrew meddles with the default configuration and this will break things, though. We shall see. --- CI/macos/before_install.amd64.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CI/macos/before_install.amd64.sh b/CI/macos/before_install.amd64.sh index 1cf403e5ea..5385338c0b 100755 --- a/CI/macos/before_install.amd64.sh +++ b/CI/macos/before_install.amd64.sh @@ -2,8 +2,8 @@ command -v /usr/local/bin/brew || arch -x86_64 bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" -arch -x86_64 bash -c "command -v ccache || /usr/local/bin/brew install ccache" -arch -x86_64 bash -c "command -v cmake >/dev/null 2>&1 || /usr/local/bin/brew install cmake" +command -v ccache >/dev/null 2>&1 || brew install ccache +command -v cmake >/dev/null 2>&1 || brew install cmake arch -x86_64 bash -c "command -v qmake >/dev/null 2>&1 && qmake -v | grep -F 'Using Qt version 6.' >/dev/null || /usr/local/bin/brew install qt@6" arch -x86_64 /usr/local/bin/brew install curl xquartz gd fontconfig freetype harfbuzz brotli openal-soft icu4c yaml-cpp sqlite From a4be773e9c6cfbac0e8faa1c3ff0091b22eb6e83 Mon Sep 17 00:00:00 2001 From: AnyOldName3 Date: Wed, 3 Sep 2025 22:04:39 +0100 Subject: [PATCH 04/12] Native ccache won't work if we insist on an architecture --- CI/macos/ccache_prep.sh | 6 +----- CI/macos/ccache_show_stats.sh | 6 +----- 2 files changed, 2 insertions(+), 10 deletions(-) diff --git a/CI/macos/ccache_prep.sh b/CI/macos/ccache_prep.sh index abd0103be0..314da2c88f 100755 --- a/CI/macos/ccache_prep.sh +++ b/CI/macos/ccache_prep.sh @@ -1,7 +1,3 @@ #!/bin/sh -ex -if [[ "${MACOS_AMD64}" ]]; then - arch -x86_64 ccache -z -M "${CCACHE_SIZE}" -else - ccache -z -M "${CCACHE_SIZE}" -fi +ccache -z -M "${CCACHE_SIZE}" diff --git a/CI/macos/ccache_show_stats.sh b/CI/macos/ccache_show_stats.sh index d7cc48c25e..75a788217f 100755 --- a/CI/macos/ccache_show_stats.sh +++ b/CI/macos/ccache_show_stats.sh @@ -1,7 +1,3 @@ #!/bin/sh -ex -if [[ "${MACOS_AMD64}" ]]; then - arch -x86_64 ccache -svv -else - ccache -svv -fi +ccache -svv From 6cdcbd957ab9be52e48621bd465eddca826f60a6 Mon Sep 17 00:00:00 2001 From: AnyOldName3 Date: Wed, 3 Sep 2025 22:06:17 +0100 Subject: [PATCH 05/12] And the same with CMake --- CI/before_script.macos.sh | 15 ++++----------- 1 file changed, 4 insertions(+), 11 deletions(-) diff --git a/CI/before_script.macos.sh b/CI/before_script.macos.sh index 3ec72cebaa..f0e4c63ba0 100755 --- a/CI/before_script.macos.sh +++ b/CI/before_script.macos.sh @@ -62,14 +62,7 @@ else ) fi -if [[ "${MACOS_AMD64}" ]]; then - arch -x86_64 cmake \ - "${CMAKE_CONF_OPTS[@]}" \ - "${BUILD_OPTS[@]}" \ - .. -else - cmake \ - "${CMAKE_CONF_OPTS[@]}" \ - "${BUILD_OPTS[@]}" \ - .. -fi +cmake \ + "${CMAKE_CONF_OPTS[@]}" \ + "${BUILD_OPTS[@]}" \ + .. From c70a7cd15b4d8a3099a0107b12f5aa80379f4e2b Mon Sep 17 00:00:00 2001 From: AnyOldName3 Date: Fri, 5 Sep 2025 01:01:41 +0100 Subject: [PATCH 06/12] Eliminate single-line ccache scripts They used to be necessary because they selected CMake of the same architecture as the build target, but now we use native ccache, they don't do anything meaningful. Make things consistent with other platforms and just call ccache directly in the CI script. --- .gitlab-ci.yml | 4 ++-- CI/macos/ccache_prep.sh | 3 --- CI/macos/ccache_show_stats.sh | 3 --- 3 files changed, 2 insertions(+), 8 deletions(-) delete mode 100755 CI/macos/ccache_prep.sh delete mode 100755 CI/macos/ccache_show_stats.sh diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index f779c52e44..6960a1d01f 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -558,7 +558,7 @@ Ubuntu_GCC_integration_tests_asan: - export CCACHE_BASEDIR="$(pwd)" - export CCACHE_DIR="$(pwd)/ccache" - mkdir -pv "${CCACHE_DIR}" - - CI/macos/ccache_prep.sh + - ccache -z -M "${CCACHE_SIZE}" - CI/before_script.macos.sh - CI/macos/build.sh - cd build @@ -578,7 +578,7 @@ Ubuntu_GCC_integration_tests_asan: s3cmd put "${dmg}" s3://openmw-artifacts/${artifactDirectory} done fi - - ../CI/macos/ccache_show_stats.sh + - ccache -svv artifacts: paths: - build/OpenMW-*.dmg diff --git a/CI/macos/ccache_prep.sh b/CI/macos/ccache_prep.sh deleted file mode 100755 index 314da2c88f..0000000000 --- a/CI/macos/ccache_prep.sh +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/sh -ex - -ccache -z -M "${CCACHE_SIZE}" diff --git a/CI/macos/ccache_show_stats.sh b/CI/macos/ccache_show_stats.sh deleted file mode 100755 index 75a788217f..0000000000 --- a/CI/macos/ccache_show_stats.sh +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/sh -ex - -ccache -svv From ff60b741a7651748be33ac91fbf750504812a7df Mon Sep 17 00:00:00 2001 From: AnyOldName3 Date: Sat, 6 Sep 2025 00:51:52 +0100 Subject: [PATCH 07/12] Move architecture-independent tool installation to common script --- CI/before_install.macos.sh | 3 +++ CI/macos/before_install.amd64.sh | 2 -- CI/macos/before_install.arm64.sh | 2 -- 3 files changed, 3 insertions(+), 4 deletions(-) diff --git a/CI/before_install.macos.sh b/CI/before_install.macos.sh index 60d2d3a38c..4d095cdb74 100755 --- a/CI/before_install.macos.sh +++ b/CI/before_install.macos.sh @@ -5,3 +5,6 @@ if [[ "${MACOS_AMD64}" ]]; then else ./CI/macos/before_install.arm64.sh fi + +command -v ccache >/dev/null 2>&1 || brew install ccache +command -v cmake >/dev/null 2>&1 || brew install cmake diff --git a/CI/macos/before_install.amd64.sh b/CI/macos/before_install.amd64.sh index 5385338c0b..16b2724fa7 100755 --- a/CI/macos/before_install.amd64.sh +++ b/CI/macos/before_install.amd64.sh @@ -2,8 +2,6 @@ command -v /usr/local/bin/brew || arch -x86_64 bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)" -command -v ccache >/dev/null 2>&1 || brew install ccache -command -v cmake >/dev/null 2>&1 || brew install cmake arch -x86_64 bash -c "command -v qmake >/dev/null 2>&1 && qmake -v | grep -F 'Using Qt version 6.' >/dev/null || /usr/local/bin/brew install qt@6" arch -x86_64 /usr/local/bin/brew install curl xquartz gd fontconfig freetype harfbuzz brotli openal-soft icu4c yaml-cpp sqlite diff --git a/CI/macos/before_install.arm64.sh b/CI/macos/before_install.arm64.sh index 6939fb3a2d..a33bc3296a 100755 --- a/CI/macos/before_install.arm64.sh +++ b/CI/macos/before_install.arm64.sh @@ -3,8 +3,6 @@ brew tap --repair brew update --quiet -command -v ccache >/dev/null 2>&1 || brew install ccache -command -v cmake >/dev/null 2>&1 || brew install cmake command -v qmake >/dev/null 2>&1 && qmake -v | grep -F "Using Qt version 6." >/dev/null || brew install qt@6 brew install curl xquartz gd fontconfig freetype harfbuzz brotli openal-soft icu4c yaml-cpp sqlite From 0a2c929a7607cd53871ce21050b61941926c517f Mon Sep 17 00:00:00 2001 From: AnyOldName3 Date: Sat, 6 Sep 2025 00:56:46 +0100 Subject: [PATCH 08/12] Don't meddle with people's git settings I can't see any reason for this to have been necessary in CI in the first place as we do CI runs for commits on a branch, not for a detached head. Despite that, I've left it in the Linux and Android scripts as we don't suggest anyone runs those on their own machines. --- CI/before_script.macos.sh | 3 --- 1 file changed, 3 deletions(-) diff --git a/CI/before_script.macos.sh b/CI/before_script.macos.sh index f0e4c63ba0..8d83949f09 100755 --- a/CI/before_script.macos.sh +++ b/CI/before_script.macos.sh @@ -1,8 +1,5 @@ #!/bin/sh -e -# Silence a git warning -git config --global advice.detachedHead false - rm -fr build mkdir build cd build From 98bb4af0cb811977215db4bfbe0cf3821d7d4811 Mon Sep 17 00:00:00 2001 From: AnyOldName3 Date: Sat, 6 Sep 2025 01:07:02 +0100 Subject: [PATCH 09/12] Use correct shebang [[ doesn't work in real sh, only in more advanced shells pretending to be sh. --- CI/before_script.macos.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CI/before_script.macos.sh b/CI/before_script.macos.sh index 8d83949f09..f8c2c74169 100755 --- a/CI/before_script.macos.sh +++ b/CI/before_script.macos.sh @@ -1,4 +1,4 @@ -#!/bin/sh -e +#!/bin/bash -e rm -fr build mkdir build From 998c738ed059127b539dcaef1c2e98b630ee0db1 Mon Sep 17 00:00:00 2001 From: AnyOldName3 Date: Sat, 6 Sep 2025 01:29:44 +0100 Subject: [PATCH 10/12] Add some options from the Windows script to the MacOS script Hopefully, they might even work, too. Also move ccache installation to the CI script as it's not mandatory to use ccache for local builds. --- .gitlab-ci.yml | 3 +- CI/before_install.macos.sh | 1 - CI/before_script.macos.sh | 94 ++++++++++++++++++++++++++++++++++++-- 3 files changed, 92 insertions(+), 6 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 6960a1d01f..23d94711f7 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -555,11 +555,12 @@ Ubuntu_GCC_integration_tests_asan: - ccache/ script: - CI/before_install.macos.sh + - brew install ccache - export CCACHE_BASEDIR="$(pwd)" - export CCACHE_DIR="$(pwd)/ccache" - mkdir -pv "${CCACHE_DIR}" - ccache -z -M "${CCACHE_SIZE}" - - CI/before_script.macos.sh + - CI/before_script.macos.sh -C - CI/macos/build.sh - cd build - for dmg in *.dmg; do mv "$dmg" "${dmg%.dmg}_${DMG_IDENTIFIER}_${CI_COMMIT_REF_NAME##*/}.dmg"; done diff --git a/CI/before_install.macos.sh b/CI/before_install.macos.sh index 4d095cdb74..aa87ad9907 100755 --- a/CI/before_install.macos.sh +++ b/CI/before_install.macos.sh @@ -6,5 +6,4 @@ else ./CI/macos/before_install.arm64.sh fi -command -v ccache >/dev/null 2>&1 || brew install ccache command -v cmake >/dev/null 2>&1 || brew install cmake diff --git a/CI/before_script.macos.sh b/CI/before_script.macos.sh index f8c2c74169..67a7e7accf 100755 --- a/CI/before_script.macos.sh +++ b/CI/before_script.macos.sh @@ -1,7 +1,69 @@ #!/bin/bash -e -rm -fr build -mkdir build +VERBOSE="" +USE_CCACHE="" +KEEP="" +USE_WERROR="" + +while [ $# -gt 0 ]; do + ARGSTR=$1 + shift + + if [ ${ARGSTR:0:1} != "-" ]; then + echo "Unknown argument $ARGSTR" + echo "Try '$0 -h'" + wrappedExit 1 + fi + + for (( i=1; i<${#ARGSTR}; i++ )); do + ARG=${ARGSTR:$i:1} + case $ARG in + V ) + VERBOSE=true ;; + + C ) + USE_CCACHE=true ;; + + k ) + KEEP=true ;; + + E ) + USE_WERROR=true ;; + + h ) + cat < Date: Sat, 6 Sep 2025 17:55:09 +0100 Subject: [PATCH 11/12] Use CCache in GitHub Actions build again --- .github/workflows/push.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/push.yml b/.github/workflows/push.yml index 4de11d7967..134a6d4033 100644 --- a/.github/workflows/push.yml +++ b/.github/workflows/push.yml @@ -82,7 +82,7 @@ jobs: max-size: 1000M - name: Configure - run: CI/before_script.macos.sh + run: CI/before_script.macos.sh -C - name: Build run: CI/macos/build.sh From fe97e3c888e1995379c2d0beb4f43002ecb4fef3 Mon Sep 17 00:00:00 2001 From: AnyOldName3 Date: Fri, 12 Sep 2025 18:09:12 +0100 Subject: [PATCH 12/12] Move brew repair and update to architecture-independent script It installs things now, so these commands are useful. --- CI/before_install.macos.sh | 3 +++ CI/macos/before_install.arm64.sh | 3 --- 2 files changed, 3 insertions(+), 3 deletions(-) diff --git a/CI/before_install.macos.sh b/CI/before_install.macos.sh index aa87ad9907..84cc99d03d 100755 --- a/CI/before_install.macos.sh +++ b/CI/before_install.macos.sh @@ -1,5 +1,8 @@ #!/bin/sh -ex +brew tap --repair +brew update --quiet + if [[ "${MACOS_AMD64}" ]]; then ./CI/macos/before_install.amd64.sh else diff --git a/CI/macos/before_install.arm64.sh b/CI/macos/before_install.arm64.sh index a33bc3296a..60217093ec 100755 --- a/CI/macos/before_install.arm64.sh +++ b/CI/macos/before_install.arm64.sh @@ -1,8 +1,5 @@ #!/bin/sh -ex -brew tap --repair -brew update --quiet - command -v qmake >/dev/null 2>&1 && qmake -v | grep -F "Using Qt version 6." >/dev/null || brew install qt@6 brew install curl xquartz gd fontconfig freetype harfbuzz brotli openal-soft icu4c yaml-cpp sqlite