Make self-contained Mac app bundle
ddeanbrown ddeanbrown I tried, I failed. Now convinced this is not possible after all. At least 2 killer problems - there is no static lib for gtk3, nor gtk4. Also the "the 'iconv_' vs 'libiconv_' namespaces macro magic" rears its ugly head - just one example, libSDL2.a has _iconv_close and libiconv.a has _libiconv_close.
Closing this ticket, not sure whether Rejected or Invalid is better.
For the lib)iconv namespace problem I think we could provide a wrapper function.
Lack of static libraries for some components would necessitate building them ourselves, and maybe that's more involved process than we want to do now?
But can you remind me what was the original reason it was necessary to do the static linking route instead of just including the libraries in the installer package? I'm guessing they had some unacceptable paths hardcoded to them? Does the reason apply also to those components that are available only as shared libraries?
Reply To cazfi
I'm guessing they had some unacceptable paths hardcoded to them?
Except that you told that homebrew works even when installed to a custom location, which sounds like there's no such dependency to specific paths. Unless the homebrew installation process does some modifications to them when installing to custom location. In which case we might be able to do the same thing.
ddeanbrown OK, I shouldn't give up so easily, will give it another try. The reason I wanted static libraries is that installing homebrew in the app bundle makes it 3.15 GB in size. After building, I can clean it out to be about 630 MB, but it's still kind of ugly and inelegant. I think in theory it could be pared down to about 300MB with static libraries (deadstrip that code, baby!). How big is the installed Windows app? But it works, so I shouldn't be so picky. The cleaning out part is the hardest task, but could be mostly automated using a script, and I've now learned enough about shell scripts to feel like I could do it well enough.
The lib)iconv namespace problem isn't a problem with freeciv code calling iconv code, it's a problem with other homebrew package static libs calling iconv code. Fixing that would mean re-building those. On top of building the missing static libraries. Which means getting the sources, which homebrew doesn't save. We don't want to go there.
With homebrew installed in the app bundle there's no problems with hardcoded paths.
Reply To ddeanbrown
OK, I shouldn't give up so easily, will give it another try. The reason I wanted static libraries is that installing homebrew in the app bundle makes it 3.15 GB in size. After building, I can clean it out to be about 630 MB, but it's still kind of ugly and inelegant. I think in theory it could be pared down to about 300MB with static libraries (deadstrip that code, baby!). How big is the installed Windows app? But it works, so I shouldn't be so picky. The cleaning out part is the hardest task, but could be mostly automated using a script, and I've now learned enough about shell scripts to feel like I could do it well enough.
I'm not convinced of the logic here. After all, the reason shared libraries exist is to make overall system size smaller by sharing one copy of the library between all executables instead of every one of them carrying a copy included to them. We do have multiple executables (in any bundle you will have at least the client and the server) So it sounds to me that you simply have not "cleaned" the shared libraries tree enough (to provide only the same things as the statically linked one. Note that when implementing any Windows packaging changes I work the opposite way; definitely not stripping stuff from complete msys2 environment, but adding only necessary parts to the installation - maybe this should work the same way.
The lib)iconv namespace problem isn't a problem with freeciv code calling iconv code, it's a problem with other homebrew package static libs calling iconv code. Fixing that would mean re-building those. On top of building the missing static libraries. Which means getting the sources, which homebrew doesn't save. We don't want to go there.
We could still provide the missing symbol, even when it's "not our fault" that it's missing. Especially with static libraries; they have no idea *where* the symbol should come from, so they would happily use the one we provide.
All this starts to sound like we should dust off the macos installation environment build scripts I did early last year (ones you wanted to replace with the new approach). Though we were aiming towards autotools build at the time, there should be not much difference from the perspective of those scripts - some adjustments to the lists of packages to install etc, but not to the overall logic.
ddeanbrown I now think you are correct about shared libraries making the overall size smaller. I also tried your way of adding the needed stuff instead of removing the un-needed stuff and that's working well. The macos installation environment build scripts you did were useful to get me started. The only problem I now have is the meson build of the qt client fails because meson can't find the qt frameworks when they live in the app bundle, which I think is a meson bug. I have opened a ticket for it - https://github.com/mesonbuild/meson/issues/11302 Will try to get autotools build of the qt client working next.
Question - I'm seeing a warning msg reported -
But I see nothing wrong with the display. Is this an ignorable warning? or is there something I need to add back in? I tried adding the adwaita-icon-theme package but that didn't help.
Reply To ddeanbrown
... Gtk-WARNING **: 20:53:50.510: Could not load a pixbuf from /org/gtk/libgtk/theme/Adwaita/assets/bullet-symbolic.svg. This may indicate that pixbuf loaders or the mime database could not be found.
I see the same warning on crosser based Windows builds of gtk clients. That latter sentence is right in that mime database is not generated at all for those installations. Some time ago I tried making the installer to generate the mime database, but the warning remained.
Seems relatively harmless also because it never warns about any other icon that this one - so doesn't look like it really had problems with loading icons in general.
ddeanbrown Success getting autotools build with qt5 to work. Both meson and autotools fail to find qt6, possibly for the same as yet unknown reason.
Unrelated problem, which I hope you can shed some light on - build errors when using meson build in the standalone. This is independent of qt. The error looks like this -
131/493 Compiling C object libfc_ai.a.p/ai_classic_classicai.c.o
FAILED: libfc_ai.a.p/ai_classic_classicai.c.o
cc -Ilibfc_ai.a.p -I. -I.. -I../dependencies/luasql/src -I../dependencies/tinycthread -I../dependencies/tolua-5.2/include -I../dependencies/cvercmp -I../utility -I../common -I../common/networking -I../common/scriptcore -I../common/aicore -I../server -I../server/advisors -I../server/scripting -I../server/generator -I../server/savegame -I../ai -I../ai/classic -I../ai/default -I/Applications/Freeciv.app/Contents/homebrew/opt/icu4c/include -I/Applications/Freeciv.app/Contents/homebrew/opt/readline/include -I/Applications/Freeciv.app/Contents/homebrew/opt/libiconv/include -I/Applications/Freeciv.app/Contents/homebrew/opt/qt@5/include -fcolor-diagnostics -Wall -Winvalid-pch -O0 -DHAVE_CONFIG_H -std=gnu17 -MD -MQ libfc_ai.a.p/ai_classic_classicai.c.o -MF libfc_ai.a.p/ai_classic_classicai.c.o.d -o libfc_ai.a.p/ai_classic_classicai.c.o -c ../ai/classic/classicai.c
In file included from ../ai/classic/classicai.c:19:
In file included from ../common/ai.h:21:
In file included from ../common/fc_types.h:174:
./specenum_gen.h:5515:17: error: implicit declaration of function 'gettext' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
names[0] = Q_(SPECENUM_VALUE0NAME);
../utility/fcintl.h:57:47: note: expanded from macro 'Q_'
#define Q_(String) skip_intl_qualifier_prefix(gettext(String))
I suspect the problem is the "C99", but how did that get there? "C99" is an option according to "man cc", which also says the default is "gnu17". I tried adding the flag "-std=gnu17", which you can see in the invocation but that didn't help. I don't get this error when building with normal homebrew, only with homebrew in the app bundle, but I can't think of how that could make a difference.
Reply To ddeanbrown
I suspect the problem is the "C99"
It's not. That would have broken all compilations for the last two decades (since compilers no longer defaulted to C89)
I'll have a look a bit later. That gettext() & Qt stuff is a bit of mess, to work around various issues especially in Windows builds.
Do you have nls enabled or disabled in that build?
Reply To ddeanbrown
I don't get this error when building with normal homebrew, only with homebrew in the app bundle, but I can't think of how that could make a difference.
Configure finding different things, and in some cases enabling features as a result (basically any configure options defaulting to "try to get this to work", and not explicit "enable" or "disable")
Compare fc_config.h of those builds to see what features they have differently (I suspect nls, but of course the compile should still work on both cases)
ddeanbrown Tried turning off the default nls setting and got a different error -
../utility/ioz.c:93:3: error: unknown type name 'lzma_stream'; did you mean 'bz_stream'?
Homebrew says "lzma is now part of the xz formula", and I have that package installed.
Comparing 2 fc_config.h files is giving me some clues, diffs in #define HAVE_LZMA_H, #define HAVE_ZSTD_H, #define HAVE_LIBINTL_H. Maybe I need more "-I..." include flags.
Reply To ddeanbrown
Comparing 2 fc_config.h files is giving me some clues, diffs in #define HAVE_LZMA_H, #define HAVE_ZSTD_H, #define HAVE_LIBINTL_H.
Does that mean that either build has one of HAVE_LZMA_H and HAVE_LIBLZMA defined, but not the other? Such combination is likely something I've never tested -> would not be surprised about bugs there.
ddeanbrown Yes there was a diff with HAVE_LZMA_H and not with HAVE_LIBLZMA - good catch.
My original build problem was due to HAVE_LIBINTL_H. I have now fixed up my include paths, learned that libintl.h is in the gettext package, and am back on track. Thanks.
Regarding the error, ./specenum_gen.h:5515:17: error: implicit declaration of function 'gettext' is invalid in C99 -Werror,-Wimplicit-function-declaration , that reminds me of a similar error message I encountered when building freeciv on macOS using MacPorts. In my case it cames from the tiny test programs which ./configure compiles. Those programs did not declare some of their functions, so the C compiler generated an implicit declaration. That implicit declaration is illegal in C99 with the -Werror and -Wimplicit-function-declaration options.
See this for more information (albeit in a MacPorts context): https://trac.macports.org/wiki/WimplicitFunctionDeclaration
Reply To jdlh
Regarding the error, ./specenum_gen.h:5515:17: error: implicit declaration of function 'gettext' is invalid in C99 -Werror,-Wimplicit-function-declaration , that reminds me of a similar error message I encountered when building freeciv on macOS using MacPorts. In my case it cames from the tiny test programs which ./configure compiles. Those programs did not declare some of their functions, so the C compiler generated an implicit declaration. That implicit declaration is illegal in C99 with the -Werror and -Wimplicit-function-declaration options.
That error (on building freeciv, not some configure test) it likely very similar to the lzma one; the main feature getting enabled at configure phase despite relevant header not being found -> compile tries to use the function despite not including the header declaring it.
Reply To cazfi
Reply To jdlh
Regarding the error, ./specenum_gen.h:5515:17: error: implicit declaration of function 'gettext' is invalid in C99 -Werror,-Wimplicit-function-declaration , that reminds me of a similar error message I encountered when building freeciv on macOS using MacPorts. In my case it cames from the tiny test programs which ./configure compiles. Those programs did not declare some of their functions, so the C compiler generated an implicit declaration. That implicit declaration is illegal in C99 with the -Werror and -Wimplicit-function-declaration options.
That error (on building freeciv, not some configure test) it likely very similar to the lzma one; the main feature getting enabled at configure phase despite relevant header not being found -> compile tries to use the function despite not including the header declaring it.
-> #46669
ddeanbrown ddeanbrown All working now. This scripting thing is new for me, so maybe some improvements could be made, but this may be good enough for a start.
ddeanbrown I tried this script again today and it hung, nothing changed on my side. Suspected a change to homebrew itself - after a couple of stabs in the dark a lucky guess fixed it. Needed to add a call to
brew developer off
New patch attached. While I was there I changed the alternate client from gtk3.22 to gtk4.
I will not miss the gtk3.22 client. Annoying thing on the Mac - when you double click on the client executable like "freeciv-qt" it launches just fine, when you double click on "freeciv-gtk3.22" it thinks ".22" is a file extension and it doesn't know what to do with a ".22" file. Arg.
ddeanbrown ddeanbrown - Need to list new files on (toplevel) Makefile.am
- The documentation still speaks of gtk3.22-client
- For main/S3_2 version setting of ack_experimental needs to be dropped (not sure if there's other differences to S3_1)
ddeanbrown ddeanbrown - Need to list new files on (toplevel) Makefile.am
Should wait on this until #46342 is done
- The documentation still speaks of gtk3.22-client
Corrected patch file attached
- For main/S3_2 version setting of ack_experimental needs to be dropped (not sure if there's other differences to S3_1)
New ticket created for this task #47585
Reply To ddeanbrown
- Need to list new files on (toplevel) Makefile.am
Should wait on this until #46342 is done
You are saying that this ticket depends on #46342?
Spun off from #46342
It looks like it is possible to build a self-contained Mac app bundle with meson, using static libraries. This would allow an end user to download a zip file, un-zip it, put freeciv.app in their Applications folder, and run it without having to install homebrew and the homebrew packages. Would need a script to build it, and a MacAppREADME.txt file to include in the zip file.
The main issue is that meson.build needs to replace (at least some) calls to c_compiler.find_library() with calls to dependency().