Maven debug and source artifacts for Derby

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Maven debug and source artifacts for Derby

Kristian Waagan-2
Hi,

A few JIRAs have been logged where users are requesting that the Derby
community provide debug artifacts for Maven (links at the end):
   o DERBY-5668: Derby does not publish source artifacts to Maven Central
   o DERBY-5543: include debug info in derby builds uploaded to maven
   o DERBY-3910: debug artifacts should be available in maven repositories

Regarding the source artifacts, I think we should just do it! Does
anyone see a reason why we don't want to do it?

The debug artifacts requires a little more consideration. There are two
requests at play here:
   a) include line numbers
   b) include the extra checks guarded by the SanityManager blocks
      (what's called a sane build)

I think many of the users would want to use artifacts (a). In fact I'd
say they should probably be the default. I imagine artifacts (b) to be
used during development and when you debug a problem. What we are
currently proving is more along the lines of (c):
   c) reduced disk-footprint JARs for small devices

(a) and (c) are suitable for production, (b) isn't because of the
performance degradation caused by the extra sanity checks.


A recent build of trunk shows the following increases in size when
including line numbers [1]:
   o derby.jar: 2.6 MB -> 3.5 MB
   o derbyclient.jar: 523 KB -> 686 KB
   o derbynet.jar: 237 KB -> 302 KB

Not compiling away the extra checks only adds a little more. The growth
is most significant for derby.jar, where it ends up at 3.7 MB.

They way I see it, adding line numbers has the following potential benefits:
   o make it easier for users to debug/investigate a problem where they
get a stack trace
   o make it easier for the Derby community to start debugging a problem
when they have a stack trace

The only downside I see is the increase in size. This probably isn't a
big deal for most users, but I expect there will be some users that want
to keep things as small as possible. As a note to this, there has been
some speculation (?) that the increased size may slow down class
loading. Again I assume this is a concern to only a subset of our users.

Together with source artifacts, I believe adding debug information will
make it easier for people using Maven as their build tool to get
involved with Derby. I think providing these artifacts can benefit both
these users and the Derby community.
I can't see how providing these artifacts would hurt.

What do others think?


Links to the issues mentioned:
https://issues.apache.org/jira/browse/DERBY-5668
https://issues.apache.org/jira/browse/DERBY-5543
https://issues.apache.org/jira/browse/DERBY-3910


Regards,
--
Kristian

[1] We're only specifying debug=true, which I assume has the same effect
as debug=on. This again I'm assuming causes -g to be passed to the
compiler, since we're not specifying debuglevel, and -g means all debug
information (-g:lines,vars,source).
Reply | Threaded
Open this post in threaded view
|

Re: Maven debug and source artifacts for Derby

Dag H. Wanvik-2
Kristian Waagan <[hidden email]> writes:

> Together with source artifacts, I believe adding debug information
> will make it easier for people using Maven as their build tool to get
> involved with Derby. I think providing these artifacts can benefit
> both these users and the Derby community.
> I can't see how providing these artifacts would hurt.

I think this question is larger than the Maven artifacts: do we want to
provide bundles that have line number debug information, i.e. your a),
in addition to b) and b) or possibly as a replacement for b) or c)?

I agree that line numbers would often by the preferred default, and the
fully stripped jars would mostly needed for constrained environments.

But we have many artifacts already, so it could increase confusion. What
do others think?

Dag
Reply | Threaded
Open this post in threaded view
|

Re: Maven debug and source artifacts for Derby

Rick Hillegas
Thanks for starting this conversation, Kristian. Here's my $0.02:

I see a number of issues here:

1) An impedance mismatch between the Derby and Maven concepts of release
artifacts.

2) The bewildering number of Derby release artifacts today.

3) A request for different Derby release artifacts.

4) A request for a source Maven artifact.

Just to be clear, I think we should try to drive our release process
toward producing fewer artifacts, not more.


------- Impedance Mismatch Between Derby and Maven ----------

For Derby, a release artifact is a zip (or tar) ball containing a
complete system of jar files which cohere with one another. For Maven, a
release artifact is a single jar file which can be mixed and matched
with other jar files at different rev levels. The two world views
diverge at a very high level:

a) For Derby, jar files are children of version numbers

b) For Maven, version numbers are children of jar files

I don't know how we go about bridging this gap. The Maven model allows
for mixing Derby jars from different versions, something which we don't
generally allow. Maybe there is a way in Maven to declare that the
version numbers of derbynet.jar and derby.jar must agree.


------- Bewildering Number of Derby Release Artifacts ----------

IMHO we already produce too many Derby artifacts. Do we really need bin
and lib artifacts? Do we really need zips and tarballs? I'd be happy
with zip-only artifacts representing the following:

i) Source (I believe this is the only artifact required by Apache)

ii) Docs, templates, and demos

iii) Bin (executable code, including jars and shell scripts)

iv) Debug-bin

That's 4 artifacts rather than the 8 we produce today (remember, we're
deprecating the Eclipse bits).


-------  Different Derby Artifacts ----------

There's a request to produce an artifact which is half-way between our
bin and debug-bin artifact, viz., executable code which includes line
numbers but no assertions. How much extra value does this add?

I think this would provide extra information for heisenbugs which don't
have repros, but in my experience we don't make much headway on those
bugs, with or without line numbers.

For bugs which have repros, we (the Derby developers who do almost all
of the bug fixing around here) see the line numbers as soon as we run
the repro, because we're all using sane builds in our sandboxes.

So who is this new artifact for? Are there other developers out there
who would be contributing fixes back to the community if we provided an
artifact with line numbers? These would be people who would take the
time to script, debug, and fix an issue. To do this, they would need to
download the source distro or set up a subversion client on one of our
branches. But they wouldn't be willing to download a debug distro.


-------  Source Maven Artifacts ----------

I don't have any objection to providing a source Maven artifact for the
purposes described by DERBY-5668. Does the source artifact need to be in
some special format to be usable by IDEs?

Thanks,
-Rick

Reply | Threaded
Open this post in threaded view
|

Re: Maven debug and source artifacts for Derby

Kristian Waagan-2
On 03.05.12 21:52, Rick Hillegas wrote:

> Thanks for starting this conversation, Kristian. Here's my $0.02:
>
> I see a number of issues here:
>
> 1) An impedance mismatch between the Derby and Maven concepts of release
> artifacts.
>
> 2) The bewildering number of Derby release artifacts today.
>
> 3) A request for different Derby release artifacts.
>
> 4) A request for a source Maven artifact.
>
> Just to be clear, I think we should try to drive our release process
> toward producing fewer artifacts, not more.

Hi Rick,

That last statement is certainly valid, but I prefer that we produce the
artifacts our users need rather than focus too much on keeping the
number down to a minimum. If the resulting process is too hard on the
community, and more specifically the release manager, we have to
simplify it with better scripting/tooling to make it bearable.
Now, actually finding out which artifacts are needed isn't necessarily easy.

Let me explain why I brought up the topic of Maven artifacts:
There are a lot of Maven users out there, and the central Maven
repository contains a huge amount of software. Further, there are
several other systems/tools that can integrate with Maven, for instance
Apache Buildr, Apache Ivy, Groovy Grape, Grails, and Scala SBT.
By having good Maven artifacts for Derby, we make it easier for people
to use Derby. If it's too hard, and I don't think it takes much to reach
this state, people may very well simply switch to another database
system - for new projects with little current investment in Derby that's
a matter of changing a few dependency definitions.

Apache is all about open-source, communities and involvement.
In my opinion, not distributing a source bundle in Maven makes it harder
for Maven users to get involved. Not including line numbers in the class
files makes it harder for someone new to understand what's going on and
to debug the code ([1]). If their application fails because of Derby,
seeing the Derby source code on line X, where the NPE happened, may
convince them that this is indeed a Derby bug and not their own "silly
mistake". The result may be that we get a JIRA report and can fix a bug.
This isn't about what these users are able to do or not, it's about how
much effort it takes for them to get involved. Too much effort means no
involvement ([2]). Derby is only one project out of a large number of
open-source projects that are fighting for involvement from contributers.
I'm not saying we'll see a flurry of new contributions if we fix these
issues, but at least we remove some obstacles for those who have a
little bit of curiosity in them.


Based on your reply I think it would be best to discuss items (2) and
(3) first. Given that we are getting close to a new release, I'm
suggesting that we let this discussion live for a while and that we
don't change anything before 10.9 goes out.

Even if the above means postponing any actions on these topics, people
are very welcome to present their opinions on these topics.


Thanks,
--
Kristian, with his community hat on :)


[1] I know we have a source zip and debug bundles on the web site, and I
know people can access the Subversion repository if they want to. But
it's more tedious than clicking "Download Sources" in my IDE and
double-clicking on the class I'm interested in.

[2] I've seen this myself, where I have given up on reporting a
[potential] bug in a piece of software because I couldn't find the
information I needed, or because filling out the bug report would take
me hours...

< snip - lots of points for discussion >