Mailman 3 February 2014 - Scons-users

Re: [Scons-users] Virtualenv and Scons?
by Larry Gadallah Nov. 19, 2015

Nov. 19, 2015

>Message: 3 >Date: Fri, 6 Sep 2013 14:16:17 -0700 >From: Bill Deegan <bill(a)baddogconsulting.com> >Subject: Re: [Scons-users] Virtualenv and Scons? >To: SCons users mailing list <scons-users(a)scons.org> >Message-ID: > <CAEyG4CFLJqtiPQoKcV8BY=t5+8+jZNhiL6Q0hikRvmzSSKAoCw(a)mail.gmail.com> >Content-Type: text/plain; charset="iso-8859-1" > >Larry, > > >So you want to run sphinx via python calls inside a SConstruct? >as in: >import sphinx >sphinx.do_something() ? > >-Bill Hello Bill: Yes, that is exactly what we want to do, except we don't want to have to have several dozen Python libraries installed in the system Python path on every build machine. We'd like to be able to create the Python environment on the fly, using virtualenv. Hence the desire to run virtualenv/pip inside a Sconstruct. Thanks, -- Larry Gadallah lgadalla(a)impinj.com x9706 >

3 11

CompilationDatabase support for SCons
by Andrew C. Morrow Jan. 20, 2015

Jan. 20, 2015

I'm interested in writing some clang tools with LibTooling for a codebase that uses SCons as its build system. LibTooling is driven by an artifact known as a CompilationDatabase, which captures state about the build so that the tool can be consistently invoked: http://clang.llvm.org/docs/JSONCompilationDatabase.html CMake has support for emitting a CompilationDatabase as a side effect of a build. I'd like to consider adding similar support to SCons. Before I start, though, a few questions: - Would such support be considered for inclusion in SCons? - If so, is anyone already working on this? I'd prefer not to duplicate the effort. - There was recently a thread here about support for considering all targets out of date. I believe such a thing would be needed to produce a compilation database as well (probably along with running in --dry-run mode). The solution in that thread was a custom Decider, but this doesn't work well if the goal is to support emitting a compilation database without users needing to customize the Environment. Would support for emitting a CompilationDatabase justify adding direct support for the "consider all out of date" feature? - What, generally, is the right place to begin? Should CompilationDatabase support be a tool? A side effect of invoking the Object and SharedObject builders? How should this be done if the goal is to support this in such a way that it continues to operate if users customize their own Object builders? Thanks, Andrew

5 15

[c++] Automatically choosing sources for linking
by Viktor Dick Aug. 27, 2014

Aug. 27, 2014

Hi. I am trying to create a SConstruct file that does not have to name any file explicitly. For this, I am restricting my project structure in the following way: the implementation of each function or method that is declared in somepath/somename.h can be found in somepath/somename.cpp (or something included therein). And every file that implements a main routine (I need to build multiple executables) starts with main_. The crucial point is to find the sources for the call to env.Program(), i.e. for each main_*.cpp file I need to find the list of .h files that are included there and I need to look if there is a .cpp file with the same name (essentially). At the moment I do this by an external script that creates this database by repeated calls to "g++ -MM" and saves it into a file which is then read from my SConstruct file. Building this database inside the SConstruct file takes too much time. I would, however, prefer some method which works within scons and automatically sees what parts of the database have to be rebuild. I could probably somehow create a .dep file for each .cpp file (inside a mirrored tree in some hidden dir) which contains the list of other .cpp files whose objects have to be also linked whenever the object of the .cpp in question is to be linked. I could make these dependent on the .cpp, so they only get rebuild when necessary. I could also create second-stage .dep2-files only for the main_*.cpp files, which are build out of the corresponding .dep-files by recursively going through them and collecting any sources that have to be linked for this executable. But can I at this stage (because any such building is done after reading the scons file if I understand correctly) collect these to give then as source parameter to the Program()-Builder? Any thanks appreciated.

5 18

Chaining generated sources.
by William Blevins May 23, 2014

May 23, 2014

Intro: I am using a custom builder to generate java file(s) via Google Proto Compiler. In most cases, one source file yields one target file in the same directory; therefore, derived-source targets can most likely be "guessed"; however, this assumption almost never works in Java. I could work around some of the issue by enforcing a 1:1 source:target relationship via the ".proto" file(s); however, in Java (at least), the target location would still be incorrect (IE. always assumed as the root package location). This should be repeatable with any N-tier (where N > 1) Java derived-source target builds where any of the following apply: 1) Class not in the root package. 2) ".proto" (or other input) creates more than 1 target. This may apply to more languages than Java, but I have no test cases readily available. If I remove the check in "SCons/tools/javac.py" in the scanner for "sourceFile.is_derived", so that the scanner will execute rather than guess, I may be able to work around this if I can force the execution of the builder immediately after construction, but I am struggling to figure out how to do this. Scons supplies an "Execute" function, but I may not be building the Action correctly. I am using a modified version of the Protoc builder written by Scott Stafford >>http://www.scons.org/wiki/ProtocBuilder Thanks, William Below are dependency tree examples regarding this issue: *** The dependency tree scons generates (INVALID) *** scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... protoc -Isrc/proto --java_out=/<stuff>/WidgetBridge/src/proto/java/proto-widget src/proto/Widget.proto src/proto/WidgetTypes.proto src/proto/CommonBaseTypes.proto javac -classpath /opt/common/protobuf/2.5.0/i686/jar/protobuf.jar -d src/proto/java/build-proto-widget -sourcepath src/proto/java/proto-widget/org/example/group/protobuf/widget src/proto/java/proto-widget/org/example/group/protobuf/widget/Widget.java src/proto/java/proto-widget/org/example/group/protobuf/widget/WidgetTypes.java src/proto/java/proto-widget/org/example/group/protobuf/common/CommonBaseTypes.java jar cf jar/protobuf-widget.jar -C src/proto/java/build-proto-widget . +-. +-jar | +-jar/protobuf-widget.jar | +-src/proto/java/build-proto-widget | | +-src/proto/java/build-proto-widget/Widget.class | | | +-src/proto/java/proto-widget/org/example/group/protobuf/widget/Widget.java | | +-src/proto/java/build-proto-widget/WidgetTypes.class | | | +-src/proto/java/proto-widget/org/example/group/protobuf/widget/WidgetTypes.java | | +-src/proto/java/build-proto-widget/CommonBaseTypes.class | | +-src/proto/java/proto-widget/org/example/group/protobuf/common/CommonBaseTypes.java | +-src/proto/java/build-proto-widget/Widget.class | | +-src/proto/java/proto-widget/org/example/group/protobuf/widget/Widget.java | +-src/proto/java/build-proto-widget/WidgetTypes.class | | +-src/proto/java/proto-widget/org/example/group/protobuf/widget/WidgetTypes.java | +-src/proto/java/build-proto-widget/CommonBaseTypes.class | +-src/proto/java/proto-widget/org/example/group/protobuf/common/CommonBaseTypes.java +-src +-src/proto +-src/proto/java +-src/proto/java/build-proto-widget | +-src/proto/java/build-proto-widget/Widget.class | | +-src/proto/java/proto-widget/org/example/group/protobuf/widget/Widget.java | +-src/proto/java/build-proto-widget/WidgetTypes.class | | +-src/proto/java/proto-widget/org/example/group/protobuf/widget/WidgetTypes.java | +-src/proto/java/build-proto-widget/CommonBaseTypes.class | +-src/proto/java/proto-widget/org/example/group/protobuf/common/CommonBaseTypes.java +-src/proto/java/proto-widget +-src/proto/java/proto-widget/org +-src/proto/java/proto-widget/org/example +-src/proto/java/proto-widget/org/example/group +-src/proto/java/proto-widget/org/example/group/protobuf +-src/proto/java/proto-widget/org/example/group/protobuf/widget | +-src/proto/java/proto-widget/org/example/group/protobuf/widget/Widget.java | +-src/proto/java/proto-widget/org/example/group/protobuf/widget/WidgetTypes.java +-src/proto/java/proto-widget/org/example/group/protobuf/common +-src/proto/java/proto-widget/org/example/group/protobuf/common/CommonBaseTypes.java scons: done building targets. *** The dependency tree scons SHOULD generate (VALID) *** scons: Reading SConscript files ... scons: done reading SConscript files. scons: Building targets ... jar cf jar/protobuf-widget.jar -C src/proto/java/build-proto-widget . +-. +-jar | +-jar/protobuf-widget.jar | +-src/proto/java/build-proto-widget | | +-src/proto/java/build-proto-widget/org | | +-src/proto/java/build-proto-widget/org/example | | +-src/proto/java/build-proto-widget/org/example/group | | +-src/proto/java/build-proto-widget/org/example/group/protobuf | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget | | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/Widget$1.class | | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/Widget$WidgetMessage$1.class | | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/Widget$WidgetMessage$Builder.class | | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/Widget.class | | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/WidgetTypes$1.class | | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/WidgetTypes$WidgetConnectionStatusType$1.class | | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/WidgetTypes$WidgetConnectionStatusType$Builder.class | | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/WidgetType.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/common | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/common/CommonBaseTypes$1.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/common/CommonBaseTypes$Common_2TupleType$1.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/common/CommonBaseTypes$Common_TimestampOrBuilder.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/common/CommonBaseTypes.class | +-src/proto/java/build-proto-widget/org | +-src/proto/java/build-proto-widget/org/example | +-src/proto/java/build-proto-widget/org/example/group | +-src/proto/java/build-proto-widget/org/example/group/protobuf | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/Widget$1.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/Widget$WidgetMessage$1.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/Widget$WidgetMessage$Builder.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/Widget.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/WidgetTypes$1.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/WidgetTypes$WidgetConnectionStatusType$1.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/WidgetTypes$WidgetConnectionStatusType$Builder.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/WidgetTypes.class | +-src/proto/java/build-proto-widget/org/example/group/protobuf/common | +-src/proto/java/build-proto-widget/org/example/group/protobuf/common/CommonBaseTypes$1.class | +-src/proto/java/build-proto-widget/org/example/group/protobuf/common/CommonBaseTypes$Common_2TupleType$1.class | +-src/proto/java/build-proto-widget/org/example/group/protobuf/common/CommonBaseTypes$Common_TimestampOrBuilder.class | +-src/proto/java/build-proto-widget/org/example/group/protobuf/common/CommonBaseTypes.class +-src +-src/proto +-src/proto/java +-src/proto/java/build-proto-widget | +-src/proto/java/build-proto-widget/org | +-src/proto/java/build-proto-widget/org/example | +-src/proto/java/build-proto-widget/org/example/group | +-src/proto/java/build-proto-widget/org/example/group/protobuf | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/Widget$1.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/Widget$WidgetMessage$1.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/Widget$WidgetMessage$Builder.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/Widget.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/WidgetTypes$1.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/WidgetTypes$WidgetConnectionStatusType$1.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/WidgetTypes$WidgetConnectionStatusType$Builder.class | | +-src/proto/java/build-proto-widget/org/example/group/protobuf/widget/WidgetTypes.class | +-src/proto/java/build-proto-widget/org/example/group/protobuf/common | +-src/proto/java/build-proto-widget/org/example/group/protobuf/common/CommonBaseTypes$1.class | +-src/proto/java/build-proto-widget/org/example/group/protobuf/common/CommonBaseTypes$Common_2TupleType$1.class | +-src/proto/java/build-proto-widget/org/example/group/protobuf/common/CommonBaseTypes$Common_TimestampOrBuilder.class | +-src/proto/java/build-proto-widget/org/example/group/protobuf/common/CommonBaseTypes.class +-src/proto/java/proto-widget +-src/proto/java/proto-widget/org +-src/proto/java/proto-widget/org/example +-src/proto/java/proto-widget/org/example/group +-src/proto/java/proto-widget/org/example/group/protobuf +-src/proto/java/proto-widget/org/example/group/protobuf/widget +-src/proto/java/proto-widget/org/example/group/protobuf/common scons: done building targets.

3 14

Packaging SCons tools
by Matthias-Christian Ott March 13, 2014

March 13, 2014

It seems all external SCons tools do not have an automated installation mechanism (some copy their files to the user's site_scons directory via Install) and with several tools it soon becomes tedious to setup, manage and replicate the site_scons directory, especially for new developers who join a team. On operating systems and distributions with proper package management software this is not a problem, you simply create a package repository and a meta-package and the installation and automatic updates become just some clicks or a single command line. On Microsoft Windows I think it would be strange to have a separate installer for each tool and without Active Directory, SCCM or similar software you have do manual updates. I thought that it would make sense to package SCons tools with distutils. Since SCons doesn't Python standard paths, one would have to generate a system specific setup.cfg or something like this to change the paths before installation [1]. I'm not sure if this is compatible with distutil's portability goals. Another option would be to abuse package_data [2]. What do you think about this? Do you have better ideas? Regards, Matthias-Christian [1] http://docs.python.org/2/install/#custom-installation [2] http://chrismiles.livejournal.com/23399.html

2 3

SConstruct return value
by Nikola Radovanovic March 1, 2014

March 1, 2014

Hi all, for some reasons (jenkins integration - dont ask me why), i need to fake result for one SConstruct - i want to always get 'build success' in SConstruct. is this possible? thanks in advance -- __________________________________________________________________________________________________________________ Nikola Radovanovic | Schneider Electric DMS NS | Smart Grid IT | SERBIA | CPASv2 team deputy Phone: +381 (0)21 488 3633 | Fax: +381 (0)21 488 3789 | Mobile: +381 (0)64 29 74 528 Email: nikola.radovanovic(a)schneider-electric-dms.com<mailto:nikola.radovanovic@schneider-electric-dms.com> | Site: www.schneider-electric-dms.com<http://www.schneider-electric-dms.com/> | Address: Narodnog fronta 25A-D, 21000 Novi Sad *** Please consider the environment before printing this e-mail

2 3

What to do when scanner is invoked for non-existing file?
by Ilya Feb. 28, 2014

Feb. 28, 2014

Hello, SCons manual in "Scanner Objects" section says the following: > Note that the file is not guaranteed to exist before the scanner is > called, so the scanner function should check that if there's any > chance that the scanned file might not exist (for example, if it's > built from other files). I have two questions about this: 1. What to do if such condition is detected? Is returning an empty list sufficient? 2. Is scanner supposed to be called again once file is created? Ilya

2 2

Re: [Scons-users] Parallel MD5
by Eric van Gyzen Feb. 26, 2014

Feb. 26, 2014

On Feb 24, 2014, at 6:33 PM, Dirk Bächle wrote: > On 22.02.2014 14:12, Eric van Gyzen wrote: >> SCons Users: >> >> It seems that SCons uses a single thread to calculate the MD5 sums of the sources, regardless of the -jN flag. Is this correct? When working in large projects with thousands of files, and only modifying one file, parallelizing the MD5 calculations across the -jN CPUs could be a major performance improvement. > > I've done some work on analysing and profiling SCons performance in the > past (see http://www.scons.org/wiki/WhySconsIsNotSlow ), and so far I > didn't find that MD5 computation is a real bottleneck. > > If you've experienced major slowdowns due to the calculation of > hashsums, I'd be very interested in your results. This includes not only > the actual figures, but also information about the used project setup > and the evaluation process. Dirk, I am embarrassed to admit that my claim was based on a hunch, with no scientific experimentation of any kind. It is complete conjecture. Thank you for your performance analysis. It is very well done. Obviously, I need to do a similar analysis with our code base. Kind regards, Eric

1 0

Parallel MD5
by Eric van Gyzen Feb. 22, 2014

Feb. 22, 2014

SCons Users: It seems that SCons uses a single thread to calculate the MD5 sums of the sources, regardless of the -jN flag. Is this correct? When working in large projects with thousands of files, and only modifying one file, parallelizing the MD5 calculations across the -jN CPUs could be a major performance improvement. Thanks in advance for the feedback, Eric

2 1

soft/hard link targets with cache area
by Jameel Shorosh Feb. 21, 2014

Feb. 21, 2014

Hi, Is there an official way to link the targets (library/binary/products etc..) to the cache area via soft/hard link instead of copying it? in our company we work in a method that we have many sub versions that shares 90% of the products and we are trying to optimize the disk space utilization via pointing target to the cache area instead of having the copy multiple times. In case there is no official way to do it we found a way to override the symbol of the method that copies it and re-implement it but I am worried if this trick may have a negative impact. please advise if the concept is correct. Jameel

2 1