Repository navigation
Duplicate symbol error emitted,when I use libiconv with -sMAIN_MODULE=1 (version:2.0.0) #11985
Description
the error list:
wasm-ld: error: duplicate symbol: __ctype_get_mb_cur_max
>>> defined in ../../lib/bitcode/libiconv.so
>>> defined in /opt/emsdk/upstream/emscripten/cache/wasm-pic/libc.a(__ctype_get_mb_cur_max.o)
wasm-ld: error: duplicate symbol: __errno_location
>>> defined in ../../lib/bitcode/libiconv.so
>>> defined in /opt/emsdk/upstream/emscripten/cache/wasm-pic/libc.a(__errno_location.o)
wasm-ld: error: duplicate symbol: __progname
>>> defined in ../../lib/bitcode/libiconv.so
>>> defined in /opt/emsdk/upstream/emscripten/cache/wasm-pic/libc.a(libc.o)
wasm-ld: error: duplicate symbol: __progname_full
>>> defined in ../../lib/bitcode/libiconv.so
>>> defined in /opt/emsdk/upstream/emscripten/cache/wasm-pic/libc.a(libc.o)
wasm-ld: error: duplicate symbol: __libc
>>> defined in ../../lib/bitcode/libiconv.so
>>> defined in /opt/emsdk/upstream/emscripten/cache/wasm-pic/libc.a(libc.o)
wasm-ld: error: duplicate symbol: __hwcap
>>> defined in ../../lib/bitcode/libiconv.so
>>> defined in /opt/emsdk/upstream/emscripten/cache/wasm-pic/libc.a(libc.o)
wasm-ld: error: duplicate symbol: __sysinfo
>>> defined in ../../lib/bitcode/libiconv.so
>>> defined in /opt/emsdk/upstream/emscripten/cache/wasm-pic/libc.a(libc.o)
wasm-ld: error: duplicate symbol: __lctrans
>>> defined in ../../lib/bitcode/libiconv.so
>>> defined in /opt/emsdk/upstream/emscripten/cache/wasm-pic/libc.a(__lctrans.o)
wasm-ld: error: duplicate symbol: __lctrans_cur
>>> defined in ../../lib/bitcode/libiconv.so
>>> defined in /opt/emsdk/upstream/emscripten/cache/wasm-pic/libc.a(__lctrans.o)
wasm-ld: error: duplicate symbol: __nl_langinfo_l
>>> defined in ../../lib/bitcode/libiconv.so
>>> defined in /opt/emsdk/upstream/emscripten/cache/wasm-pic/libc.a(langinfo.o)
wasm-ld: error: duplicate symbol: __nl_langinfo
>>> defined in ../../lib/bitcode/libiconv.so
>>> defined in /opt/emsdk/u (You can run make with `VEBOpstream/emscripten/cache/wasm-pic/libc.a(langinfo.o)
wasm-ld: error: duplicate symbol: pthread_self
>>> defined in ../../lib/bitcode/libiconv.so
>>> defined in /opt/emsdk/upstream/emscripten/cache/wasm-pic/libc.a(emscripten_pthread.o)
wasm-ld: error: duplicate symbol: __emscripten_pthread_data_constructor
>>> defined in ../../lib/bitcode/libiconv.so
>>> defined in /opt/emsdk/upstream/emscripten/cache/wasm-pic/libc.a(emscripten_pthread.o)
wasm-ld: error: duplicate symbol: __fsmu8
>>> defined in ../../lib/bitcode/libiconv.so
>>> defined in /opt/emsdk/upstream/emscripten/cache/wasm-pic/libc.a(internal.o)
wasm-ld: error: duplicate symbol: mbrtowc
>>> defined in ../../lib/bitcode/libiconv.so
>>> defined in /opt/emsdk/upstream/emscripten/cache/wasm-pic/libc.a(mbrtowc.o)
libiconv.so:
is compiled used emconfigure ./configure, version is 1.16.0
However, when set -sMAIN_MODULE=0 ,everything is ok!
Can you help me to solve this ploblem? I want to use dynamic link in my main program. but i do not know how to solve the porblem.
Activity
sbc100 commentedon Aug 21, 2020
It looks like your "shared library" contains the contents of libc.a.. normally this doesn't matter since the linker will ignore the second copy of libc.a.
However, one of the side effects of
-s MAIN_MODULE=1is that the whole link command get wrapped in--whole-archivewhich force libc.a to be included a second time.Your best bet is to configure and build without shared libraries and use static libraries instead. i.e. you should be linking against
libiconv.a. I believe the--disable-sharedargument to configure will do this.sbc100 commentedon Aug 21, 2020
Basically it seems the
MAIN_MODULEis not compaitble with out pseudo.sofiles.RReverser commentedon Jun 7, 2021
I think I'm running into the same issue with gphoto2.
In this case using static libraries is not an option, because it heavily depends on libltdl to load modules dynamically.
Strangely,
-sMAIN_MODULE=0tip from the issue description did let it compile but I'm yet to check what this means for runtime (I suppose it wouldn't be able to load side modules?).sbc100 commentedon Jun 7, 2021
I'm curious to see the full link commands used for libgphoto2_port.so and libltdl.so in the failing case. Can you include those if are able to reproduce again?
From the error message it looks like libgphoto2_port.so and libltdl.so are both actually single object files (not actually side modules). Its is important to distinguish bettween actual shared libraries (side modules) and the intermediate static objects that emscripten produces when you build with
-shared. In neither case should we be including libc.a (strcmp etc) so there is clearly a bug here.RReverser commentedon Jun 7, 2021
Perhaps I need to clean & recompile them with explicit
LDFLAGS='-s SIDE_MODULE=1'? I thought building shared .so would do that automatically, but I don't know much about autoconf / emmake / make integration.Let me try again and I'll share the commands.
RReverser commentedon Jun 7, 2021
Yeah
-s SIDE_MODULE=1doesn't seem to help. This is what I used:$ emmake make install LDFLAGS='-s SIDE_MODULE=1'For gphoto I used (last iteration):
RReverser commentedon Jun 7, 2021
Ah, sorry, forgot to add. The actual link command that fails looks like this:
sbc100 commentedon Jun 7, 2021
The link command that produced
/usr/local/google/home/rreverser/libtool-2.4.6/installed/lib/libltdl.sowould be really useful too.RReverser commentedon Jun 7, 2021
Hm, looks like it's generated by make -> libtool -> .so, and I'm not sure yet how to get it to show the final command...
sbc100 commentedon Jun 7, 2021
If you remove the file (perhaps remove libltdl.*) and then run make again it should print the full command by default. If not, the libtool command is fine too.
RReverser commentedon Jun 8, 2021
It still didn't print the full command, but in the end I found that I can get libltdl to emit a static library, and use it in gphoto. It's the libs that gphoto uses with libltdl that have to be dynamic and... I'm running into the same issues with those. Except this time I can't just change them to be static libraries.
Luckily, this Makefile prints full commands:
In this build I haven't added
SIDE_MODULE/MAIN_MODULEyet as it would require very careful review of the build system to add them in the right places... I wonder if one of those approaches could work:SIDE_MODULEwhen something is built with-sharedon the Emscripten side. Is there any reason it doesn't? In theory, that would be closest to proper shared libs.wasm-ldto just ignore duplicate symbols somehow, so that linking would behave sort of like it does for.aarchives - that is, only one symbol wins when two.sos happened to link in the same library?RReverser commentedon Jun 8, 2021
If I add
SIDE_MODULE, then I still get conflict like above but between different libs (libgphoto2_port.soandlibltdl.ainstead oflibgphoto2_port.soandlibgphoto2.so):hedwiggggg commentedon Sep 5, 2021
Sorry, this is a little bit offtopic, but I wanted to ask, if someone of you managed to port libgphoto2 to webassembly? I would love to be able to use libgphoto2 as wasm in nodejs without writing a node native addon. It seems to be possible at least (https://github.com/odejesush/webusb-wasm-libgphoto2) but unfortunately there are no instructions and i am a bloody beginner in the C world. Maybe one of you can help me, that would be great.
@sbc100 @RReverser @ucanudo
RReverser commentedon Sep 5, 2021
Stay tuned, I'm going to write something up soon-ish :)
UPD: For other looking for this - it's up now. https://web.dev/porting-libusb-to-webusb/
pepijndevos commentedon Jan 21, 2022
I'm running into what seems to be the same bug in capnproto:
omitting most of the build, it ends like so
sbc100 commentedon Jan 21, 2022
The problem is that these shared libraries are actually just object files : https://emscripten.org/docs/compiling/Building-Projects.html#faux-dynamic-linking
If you like libc and libc++ into those object you will get duplicate symbols. Do you know how this arguments ended up the link line:
-lstubs -lc -lcompiler_rt -lc++-noexcept -lc++abi-noexcept -ldlmalloc -lc_rt -lsocketsfor your shared objects? If you can remove those I think it could solve you issue.Better still, can you disable dynamic libraries completely and just use static libraries? Passing
--disable-sharedto configure might be enough to do this.pepijndevos commentedon Jan 22, 2022
So is it working as intended, or is it indeed a bug for libc symbols to get into the build?
--disable-sharedactually seemed to help. Now I'm getting missing symbol errors instead.All of them originate in libkj's exception stuff. I tried building with
-fno-rtti -fno-exceptionsbut no change.sbc100 commentedon Jan 22, 2022
Its an artifact/side-effect/bug due to the way we do fake dyanmic libraries. Normally its not an issue because libc won't be included .. but your build seems to be explicitly asking for libc to be included by adding
-lcon the command line (note that most programs should never need to add-lcexplicitly.. unless they are doing something like-nostdlib). We could ignore the request to explicitly include libc and other system libraries when building fake shared libraries... but honestly--disable-sharedis the far better solution in this case.sbc100 commentedon Jan 22, 2022
Regarding the missing symbols:
sigaltstack: Emscripten doesn't have any support for this today. We could possible add some kind of dummy/stub implementation but it would be better if you could avoid depending on it. Does capnproto have some kind of way to avoid this? HAVE_SIGNALSTACK?fallocate: It looks like we currently only supportposix_fallocatealthough it looks like we could easily add support forfallocateby just adding this file to our libc build (its currently excluded): https://github.com/emscripten-core/emscripten/blob/main/system/lib/libc/musl/src/linux/fallocate.c__cxa_get_globals: I'm not sure about this one .. I would need to take a look at exception.o and see why it might be referencing that symbol. I've not previously seen that symbol being referenced out side of libc++abi itself.