Problems compiling MonetDB-11.11.5 on SuSE SLES-11 SP2

[Jukka Santala]

SuSE Linux Enterprise Server 11 SP2, release in March. As already
covered on the bug-tracker etc. old bison won't work, so I had to
install latest version, 2.6.2 to build.
After that:
make[5]: Entering directory `/opt/MonetDB/MonetDB-11.11.5/sql/server'
/bin/sh ../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I.
-I../.. -I. -I../include -I./../include -I../common -I./../common
-I../storage -I./../storage -I../../clients/mapilib
-I./../../clients/mapilib -I../../common/options
-I./../../common/options -I../../common/stream -I./../../common/stream
-I../../gdk -I./../../gdk -DLIBSQLSERVER -g -O2 -c -o
libsqlserver_la-sql_scan.lo `test -f 'sql_scan.c' || echo './'`sql_scan.c
libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../.. -I. -I../include
-I./../include -I../common -I./../common -I../storage -I./../storage
-I../../clients/mapilib -I./../../clients/mapilib -I../../common/options
-I./../../common/options -I../../common/stream -I./../../common/stream
-I../../gdk -I./../../gdk -DLIBSQLSERVER -g -O2 -c sql_scan.c -fPIC
-DPIC -o .libs/libsqlserver_la-sql_scan.o
In file included from sql_semantic.h:27,
from sql_scan.c:29:
sql_parser.h:191: error: conflicting types for 'sqlparse'
y.tab.h:678: error: previous declaration of 'sqlparse' was here

No idea how to properly fix that, but added "#define YYPARSE_PARAM parm"
at the front of sql_scan.c, which forces the prototype to retain the
parameter.

Next problem, HowToStart.rst states: "Then in any directory (preferably
a *new, empty* directory and *not* in the ``MonetDB`` top-level
directory) give the command:: .../configure [<options>]", which is
indeed recommended practice.
Unfortunately, nobody has apparently ever done this. If configure and
make are ran in a different directory from the source root, "make check"
will fail to find monetdb5/modules/mal/mal_init.mal and who knows what
else, giving long time of head-scratching.

After re-doing configure and make in the prohibited directory, make
check will report:
.
opt_sql_append : Crashed!

* (opt_sql_append.test.out.FILTERED) significantly

* (opt_sql_append.test.err.FILTERED) significantly
.......................................................................................................................................................................................................................................
* (recycle04.test.out.FILTERED) significantly
...........................................................................................................................................................................................................................................................................
!ERROR: Testing FAILED SIGNIFICANTLY (2 out of 499 tests failed)

1 out of 499 tests resulted in a crash
extras/mal_optimizer_template/opt_sql_append
1 out of 499 tests produced SIGNIFICANTLY different output
mal/recycle04

To be frank, that sounds "significantly" like something translated by
Google translator to/from some lesser known language. No idea what it's
trying to tell me, or how to proceed from there. Can't find any Google
search hits on the problems either. Suggestions?

-Jukka Santala

[Stefan Manegold]

Hi,

Post by Jukka Santala
SuSE Linux Enterprise Server 11 SP2, release in March. As already
covered on the bug-tracker etc. old bison won't work, so I had to
install latest version, 2.6.2 to build.

Are you using HG sources or a released source tarball?
The latter is recommended and should not require bison at all ...

Post by Jukka Santala
make[5]: Entering directory `/opt/MonetDB/MonetDB-11.11.5/sql/server'
/bin/sh ../../libtool --tag=CC --mode=compile gcc -DHAVE_CONFIG_H -I.
-I../.. -I. -I../include -I./../include -I../common -I./../common
-I../storage -I./../storage -I../../clients/mapilib
-I./../../clients/mapilib -I../../common/options
-I./../../common/options -I../../common/stream -I./../../common/stream
-I../../gdk -I./../../gdk -DLIBSQLSERVER -g -O2 -c -o
libsqlserver_la-sql_scan.lo `test -f 'sql_scan.c' || echo './'`sql_scan.c
libtool: compile: gcc -DHAVE_CONFIG_H -I. -I../.. -I. -I../include
-I./../include -I../common -I./../common -I../storage -I./../storage
-I../../clients/mapilib -I./../../clients/mapilib -I../../common/options
-I./../../common/options -I../../common/stream -I./../../common/stream
-I../../gdk -I./../../gdk -DLIBSQLSERVER -g -O2 -c sql_scan.c -fPIC
-DPIC -o .libs/libsqlserver_la-sql_scan.o
In file included from sql_semantic.h:27,
sql_parser.h:191: error: conflicting types for 'sqlparse'
y.tab.h:678: error: previous declaration of 'sqlparse' was here
No idea how to properly fix that, but added "#define YYPARSE_PARAM parm"
at the front of sql_scan.c, which forces the prototype to retain the
parameter.
Next problem, HowToStart.rst states: "Then in any directory (preferably
a *new, empty* directory and *not* in the ``MonetDB`` top-level
directory) give the command:: .../configure [<options>]", which is
indeed recommended practice.
Unfortunately, nobody has apparently ever done this.

I can ensure you, we do this, and only this several times per day, so you
can assume that building MonetBD this way is very well tested ;-)

Post by Jukka Santala
If configure and
make are ran in a different directory from the source root, "make check"
will fail to find monetdb5/modules/mal/mal_init.mal and who knows what
else, giving long time of head-scratching.

Ok, I/we do admit that `make check` has not been used by us for a long time
(at least not by myself), and thus, it might not work properly anymore.
Hence, thank you very much for notifing us about it.
We will either fix it, or remove it completely.

We test MonetDB by running `Mtest.py`, (for now still?) separately in
.../MonetDB/clients/ , .../MonetDB/monetdb5/ , .../MonetDB/sql/ , and
.../MonetDB/geom/

Post by Jukka Santala
After re-doing configure and make in the prohibited directory, make
.
opt_sql_append : Crashed!
* (opt_sql_append.test.out.FILTERED) significantly
* (opt_sql_append.test.err.FILTERED) significantly
.......................................................................................................................................................................................................................................
* (recycle04.test.out.FILTERED) significantly
...........................................................................................................................................................................................................................................................................
!ERROR: Testing FAILED SIGNIFICANTLY (2 out of 499 tests failed)
1 out of 499 tests resulted in a crash
extras/mal_optimizer_template/opt_sql_append
1 out of 499 tests produced SIGNIFICANTLY different output
mal/recycle04
To be frank, that sounds "significantly" like something translated by
Google translator to/from some lesser known language.

Well, we don't need to understand this comment, do we?

Post by Jukka Santala
No idea what it's trying to tell me,

MonetDB testing is mainly for internal/development use.
The existence of a "quasi standard", without further documentation than
(from HowToStart.rst)
"
This will perform a large number of tests, some are unfortunately
still expected to fail, but most should go successfully.
"
might suggest differently. We'll consider making our documentation even more
explicit on this.
Thus, the above just tells you this: some of the tests did fail in your case.
Most of the also fail with our nightly testing, cf.,
http://monetdb.cwi.nl/testweb/web/status.php?branch=Jul2012
and thus are not considered crucial.

Post by Jukka Santala
or how to proceed from there.

Simply enjoy using MonetDB ;-)

Post by Jukka Santala
Can't find any Google
search hits on the problems either. Suggestions?

See above ;-)

Thanks again, for trying all this and notifying us. Given that we never had
any feedback on it before, we were not aware of the fact that people outside
the core MonetDb developers team were using MonetDB's testing facilities,
thus, we never spent more time on improving their documentation or making
them more end-user friendly. We'll take your comments as tirgger to
reconsiders this in some future release.

Best,
Stefan

Post by Jukka Santala
-Jukka Santala
------------------------------------------------------------------------------
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
_______________________________________________
Monetdb-developers mailing list
https://lists.sourceforge.net/lists/listinfo/monetdb-developers

[Fabian Groffen]

Hi,

Post by Jukka Santala
SuSE Linux Enterprise Server 11 SP2, release in March. As already
covered on the bug-tracker etc. old bison won't work, so I had to
install latest version, 2.6.2 to build.
In file included from sql_semantic.h:27,
sql_parser.h:191: error: conflicting types for 'sqlparse'
y.tab.h:678: error: previous declaration of 'sqlparse' was here
No idea how to properly fix that, but added "#define YYPARSE_PARAM parm"
at the front of sql_scan.c, which forces the prototype to retain the
parameter.

This should be fixed in Jul2012-SP1 which should be released very soon
from now (today?). We run nightly building with this version of bison
without a problem.

See for example
http://monetdb.cwi.nl/testweb/logs/45208:410d9805232a/GNU-Darwin-x86_64-propcheck/MdoClient.identify
http://monetdb.cwi.nl/testweb/logs/45208:410d9805232a/GNU-Darwin-x86_64-propcheck/configure
http://monetdb.cwi.nl/testweb/logs/45208:410d9805232a/GNU-Darwin-x86_64-propcheck/make

And the fixes in changesets d3e0e0e7e4f6 and 5df311c1b87f.

[Jukka Santala]

Post by Fabian Groffen
This should be fixed in Jul2012-SP1 which should be released very soon
from now (today?). We run nightly building with this version of bison
without a problem. See for example
http://monetdb.cwi.nl/testweb/logs/45208:410d9805232a/GNU-Darwin-x86_64-propcheck/MdoClient.identify
http://monetdb.cwi.nl/testweb/logs/45208:410d9805232a/GNU-Darwin-x86_64-propcheck/configure
http://monetdb.cwi.nl/testweb/logs/45208:410d9805232a/GNU-Darwin-x86_64-propcheck/make
And the fixes in changesets d3e0e0e7e4f6 and 5df311c1b87f.

Thank you, I see the fix is more involved than just matching the
function prototype.
The SP1 version is on the ftp-sites now, and I can confirm it compiles
as-is SuSE SLES-11 SP2 now,
though 'make check' still needs to be run after building in the source
root, and after 'make install'.

However, 'make check' now reports:
make check-TESTS
make[5]: Entering directory `/opt/MonetDB/MonetDB-11.11.7/monetdb5'

Mtest.py Warning: No tests found in
'/opt/MonetDB/MonetDB-11.11.7/monetdb5/extras/compiler/Tests`; skipping
directory!

.
* (json00.test.out.FILTERED) significantly
.
* (json01.test.out.FILTERED) significantly
.
(cut here for brevity)
.
opt_sql_append : Crashed!

* (opt_sql_append.test.out.FILTERED) significantly

* (opt_sql_append.test.err.FILTERED) significantly
..................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................................
!ERROR: Testing FAILED SIGNIFICANTLY (40 out of 538 tests failed)

1 out of 538 tests resulted in a crash
39 out of 538 tests produced SIGNIFICANTLY different output

cat `find -name json00.test.out.FILTERED` reveals:

# 12:08:55 > "mserver5" "--debug=10" "--set" "gdk_nr_threads=0" "--set"
"monet_mod_path=/opt/MonetDB/MonetDB-11.11.7/sql/backends/monet5/.libs:/opt/MonetDB/MonetDB-11.11.7/sql/backends/monet5:/opt/MonetDB/MonetDB-11.11.7/sql/backends/monet5/vaults/.libs:/opt/MonetDB/MonetDB-11.11.7/sql/backends/monet5/vaults:/opt/MonetDB/MonetDB-11.11.7/sql/sql:/opt/MonetDB/MonetDB-11.11.7/geom/monetdb5/.libs:/opt/MonetDB/MonetDB-11.11.7/geom/monetdb5:/opt/MonetDB/MonetDB-11.11.7/monetdb5/optimizer:/opt/MonetDB/MonetDB-11.11.7/monetdb5/optimizer:/opt/MonetDB/MonetDB-11.11.7/monetdb5/scheduler:/opt/MonetDB/MonetDB-11.11.7/monetdb5/scheduler:/opt/MonetDB/MonetDB-11.11.7/monetdb5/modules/atoms:/opt/MonetDB/MonetDB-11.11.7/monetdb5/modules/atoms:/opt/MonetDB/MonetDB-11.11.7/monetdb5/modules/kernel:/opt/MonetDB/MonetDB-11.11.7/monetdb5/modules/kernel:/opt/MonetDB/MonetDB-11.11.7/mo
netdb5/modules/mal:/opt/MonetDB/MonetDB-11.11.7/monetdb5/modules/mal:/opt/MonetDB/MonetDB-11.11.7/monetdb5/extras/compiler:/opt/MonetDB/MonetDB-11.11.7/monetdb5/extras/compiler"
"--set" "gdk_dbfarm=/opt/MonetDB/MonetDB-11.11.7/monetdb5/dbfarm"
"--set" "mapi_open=true" "--set" "mapi_port=32199" "--set"
"monet_prompt=" "--trace" "--forcemito" "--set" "mal_listing=2"
"--dbname=mTests_extras_jaql" "json00.mal"
# 12:08:55 >

# MonetDB 5 server v11.11.7 "Jul2012-SP1"
# Serving database 'mTests_extras_jaql', using 24 threads
# Compiled for x86_64-unknown-linux-gnu/64bit with 64bit OIDs
dynamically linked
# Found 62.908 GiB available main-memory.
# Copyright (c) 1993-July 2008 CWI.
# Copyright (c) August 2008-2012 MonetDB B.V., all rights reserved
# Visit http://www.monetdb.org/ for further information
# Listening for connection requests on mapi:monetdb://query:32199/
# MonetDB/GIS module loaded
# MonetDB/SQL module loaded
!TypeException:user.main[2]:'json.shred' undefined in: (j1:any, j2:any,
j3:any, j4:any, j5:any, j6:any, j7:any) := json.shred(_9:str)
!TypeException:user.main[3]:'json.print' undefined in: _10:any :=
json.print(s:streams, j1:any, j2:any, j3:any, j4:any, j5:any, j6:any,
j7:any)
!TypeException:user.main[4]:'json.shred' undefined in: (j1:any, j2:any,
j3:any, j4:any, j5:any, j6:any, j7:any) := json.shred(_11:str)
!TypeException:user.main[5]:'json.print' undefined in: _12:any :=
json.print(s:streams, j1:any, j2:any, j3:any, j4:any, j5:any, j6:any,
j7:any)
!TypeException:user.main[6]:'json.shred' undefined in: (j1:any, j2:any,
j3:any, j4:any, j5:any, j6:any, j7:any) := json.shred(_13:str)
!TypeException:user.main[7]:'json.print' undefined in: _14:any :=
json.print(s:streams, j1:any, j2:any, j3:any, j4:any, j5:any, j6:any,
j7:any)
!TypeException:user.main[8]:'json.shred' undefined in: (j1:any, j2:any,
j3:any, j4:any, j5:any, j6:any, j7:any) := json.shred(_15:str)
!TypeException:user.main[9]:'json.print' undefined in: _16:any :=
json.print(s:streams, j1:any, j2:any, j3:any, j4:any, j5:any, j6:any,
j7:any)
!TypeException:user.main[11]:'json.load' undefined in: (i1:any, i2:any,
i3:any, i4:any, i5:any, i6:any, i7:any) := json.load(_25:str)
!TypeException:user.main[15]:'json.store' undefined in: _29:any :=
json.store(_25:str, j1:any, j2:any, j3:any, j4:any, j5:any, j6:any, j7:any)
!TypeException:user.main[16]:'json.load' undefined in: (i1:any, i2:any,
i3:any, i4:any, i5:any, i6:any, i7:any) := json.load(_25:str)
!TypeException:user.main[17]:'json.print' undefined in: _30:any :=
json.print(s:streams, i1:any, i2:any, i3:any, i4:any, i5:any, i6:any,
i7:any)
!TypeException:user.main[18]:'json.drop' undefined in: _31:any :=
json.drop(_25:str)
function user.main():void;

So I'm guessing monet_mod_path doesn't get set properly. The
opt_sql_append errors which I can't make sense out of are still there as
well. And I still think the 'make check' output could be bit more clear
about what it is doing that some dots and "* (json00.test.out.FILTERED)
significantly" :)

-Jukka Santala

[Fabian Groffen]

Post by Jukka Santala

Post by Fabian Groffen
This should be fixed in Jul2012-SP1 which should be released very soon
from now (today?). We run nightly building with this version of bison
without a problem. See for example
http://monetdb.cwi.nl/testweb/logs/45208:410d9805232a/GNU-Darwin-x86_64-propcheck/MdoClient.identify
http://monetdb.cwi.nl/testweb/logs/45208:410d9805232a/GNU-Darwin-x86_64-propcheck/configure
http://monetdb.cwi.nl/testweb/logs/45208:410d9805232a/GNU-Darwin-x86_64-propcheck/make
And the fixes in changesets d3e0e0e7e4f6 and 5df311c1b87f.

Thank you, I see the fix is more involved than just matching the
function prototype.
The SP1 version is on the ftp-sites now, and I can confirm it compiles
as-is SuSE SLES-11 SP2 now,
though 'make check' still needs to be run after building in the source
root, and after 'make install'.

Yes, we should remove make check. In any case it doesn't do what the
general contract of `make check' implies.

Please report further bugs on our bugtracking system at
bugs.monetdb.org.