3.3.1 to 4.0 migration

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

3.3.1 to 4.0 migration

Helge Weissig
Our code base is heavily invested in some of the functionality removed from torque 4.0 and I was wondering if any one had some advice on a migration path:

We make extensive use of village records obtained via BasePeer.doSelect(Criteria), for example. Those results could probably be considered a view of the data and I am wondering if that would be the correct/recommended/best approach.
We have added our own caching implementation (configurable via the schema definition) to the Object.vm and Peer.vm templates. I have read through the new generator documentation and the customization part of the OR Mapping Reference (https://db.apache.org/torque/torque-4.0/documentation/orm-reference/customizing-generation.html <https://db.apache.org/torque/torque-4.0/documentation/orm-reference/customizing-generation.html>) but I don’t seem to be able to find the relevant information to put it all together. Basically, I would like to override one or more templates. I think I know how to specify that in an outlet, but do I also need the control configuration in the conf directory? The maven plugin (we use maven) has configuration parameters for overrideConfigDir and overrideConfigPackage… do I need to set those?
This is more of a maven question, maybe, alas I have the feeling it isn’t actually possible to implement through it, but is there a way to skip the code generation steps if the schema sources have not changed?
I know how to configure logging for the generator, but the switch between loglevel WARN and INFO is quite severe. Is there a way to log at INFO level but only to a file, not to the console? We use log4j throughout our project.

Thanks very much in advance for any advice and pointers!

cheers,
h.

Helge Weissig, Ph.D.
Director, Bioinformatics & IT
ActivX Biosciences, Inc.
www.activx.com <http://www.activx.com/>
www.kinativ.com <http://www.kinativ.com/>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3.3.1 to 4.0 migration

Helge Weissig
I forgot about the HTML filter this group uses… pardon the crappy formatting. Here is a text-only version:

Our code base is heavily invested in some of the functionality removed from torque 4.0 and I was wondering if any one had some advice on a migration path:

        1. We make extensive use of village records obtained via BasePeer.doSelect(Criteria), for example. Those results could probably be considered a view of the data and I am wondering if that would be the correct/recommended/best approach.

        2. We have added our own caching implementation (configurable via the schema definition) to the Object.vm and Peer.vm templates. I have read through the new generator documentation and the customization part of the OR Mapping Reference (https://db.apache.org/torque/torque-4.0/documentation/orm-reference/customizing-generation.html) but I don’t seem to be able to find the relevant information to put it all together. Basically, I would like to override one or more templates. I think I know how to specify that in an outlet, but do I also need the control configuration in the conf directory? The maven plugin (we use maven) has configuration parameters for overrideConfigDir and overrideConfigPackage… do I need to set those?

        3. This is more of a maven question, maybe, alas I have the feeling it isn’t actually possible to implement through it, but is there a way to skip the code generation steps if the schema sources have not changed?

        4. I know how to configure logging for the generator, but the switch between loglevel WARN and INFO is quite severe. Is there a way to log at INFO level but only to a file, not to the console? We use log4j throughout our project.

Thanks very much in advance for any advice and pointers!

cheers,
h.

> On Jul 1, 2015, at 9:51 AM, Helge Weissig <[hidden email]> wrote:
>
> Our code base is heavily invested in some of the functionality removed from torque 4.0 and I was wondering if any one had some advice on a migration path:
>
> We make extensive use of village records obtained via BasePeer.doSelect(Criteria), for example. Those results could probably be considered a view of the data and I am wondering if that would be the correct/recommended/best approach.
> We have added our own caching implementation (configurable via the schema definition) to the Object.vm and Peer.vm templates. I have read through the new generator documentation and the customization part of the OR Mapping Reference (https://db.apache.org/torque/torque-4.0/documentation/orm-reference/customizing-generation.html <https://db.apache.org/torque/torque-4.0/documentation/orm-reference/customizing-generation.html>) but I don’t seem to be able to find the relevant information to put it all together. Basically, I would like to override one or more templates. I think I know how to specify that in an outlet, but do I also need the control configuration in the conf directory? The maven plugin (we use maven) has configuration parameters for overrideConfigDir and overrideConfigPackage… do I need to set those?
> This is more of a maven question, maybe, alas I have the feeling it isn’t actually possible to implement through it, but is there a way to skip the code generation steps if the schema sources have not changed?
> I know how to configure logging for the generator, but the switch between loglevel WARN and INFO is quite severe. Is there a way to log at INFO level but only to a file, not to the console? We use log4j throughout our project.
>
> Thanks very much in advance for any advice and pointers!
>
> cheers,
> h.
>
> Helge Weissig, Ph.D.
> Director, Bioinformatics & IT
> ActivX Biosciences, Inc.
> www.activx.com <http://www.activx.com/>
> www.kinativ.com <http://www.kinativ.com/>


---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3.3.1 to 4.0 migration

Thomas Fox
Helge Weissig wrote:

> I forgot about the HTML filter this group uses… pardon the crappy formatting.
> Here is a text-only version:
>
> Our code base is heavily invested in some of the functionality removed from torque 4.0
> and I was wondering if any one had some advice on a migration path:
>
> 1. We make extensive use of village records obtained via
>          BasePeer.doSelect(Criteria), for example.
>          Those results could probably be considered a view of the data
>          and I am wondering if that would be the correct/recommended/best approach.

If you are using this to read data only, have a look at the RecordMapper functionality. A record mapper maps DB Columns to an object. You can use different record mappers for the same table (e.g. it can be passed in through SomePeer.doSelect(Criteria, RecordMapper<TT>), so have different object representations. For a generic village-record-like representation, see org.apache.torque.om.mapper.ObjectListMapper (personally I do not like this representation, put perhaps it serves your needs). Views (now supported !) are also a more db-centric option.

> 2. We have added our own caching implementation
>          (configurable via the schema definition) to the Object.vm
>          and Peer.vm templates. I have read through the new generator documentation
>          and the customization part of the OR Mapping Reference
>          (https://db.apache.org/torque/torque-4.0/documentation/orm-reference/customizing-generation.html)
>          but I don’t seem to be able to find the relevant information
>          to put it all together. Basically, I would like to override one or more templates.
>          I think I know how to specify that in an outlet, but do I also need the control configuration
>          in the conf directory? The maven plugin (we use maven) has configuration parameters
>          for overrideConfigDir and overrideConfigPackage… do I need to set those?

Have you checked the code-gen tutorial (http://db.apache.org/torque/torque-4.0/documentation/tutorial/orm/index.html) for a basic understanding ?
Also, there is an example in the torque test project (https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/src/main/torque-gen) which basically does what you want, i.e. overrides the Peer template (adding @SuppressWarnings annotations). The torque generation config which invokes this can be found in https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/pom.xml lines 183ff.
To be more specific:
Yes, you need at least an empty control configuration.
You need to set either overrideConfigDir or overrideConfigPackage, depending whether the additional templates can be found in the file sytsem or in the classpath.
Please ask again if you cannot get it working. Although more complicated :-(, the new generator is much more extensible than the old one. In the current trunk, you can even use groovy templates :-)

> 3. This is more of a maven question, maybe, alas I have the feeling
>          it isn’t actually possible to implement through it, but is there a way
>          to skip the code generation steps if the schema sources have not changed?

Sorry, this is not any more possible. The problem is to reliably find out whether the file has changed (e.g. some OS do not change the change date of a file when it is copied). I have seen more than one situation where schema changes did not get picked up because of this option, so my personal recommendation is not to rely on such a mechanism. Also, mvn generate is typically not executed often while developing, so long generation times should not slow down development too much.
However, if you still want to use it, you can patch the Mojo class in the Torque maven plugin, this should not be too difficult (please share if you choose to do it).

> 4. I know how to configure logging for the generator, but the switch between loglevel
>          WARN and INFO is quite severe. Is there a way to log at INFO level but only to a file,
>          not to the console? We use log4j throughout our project.

Yes, check the log4j configuration documentation. Basically you need to override the log4j configuration file and use two appenders, one file appender and one console appender, with different log levels. The log4j configuration needs to be in the classpath /org/apache/torque/generator/log4j.properties and it must have a higher classpath priority than the torque-generator.jar (I am currently not sure how to achieve this using maven, the last resort is patching the generator)

Good luck!

   Thomas

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3.3.1 to 4.0 migration

Helge Weissig
Thanks Thomas! Some comments and follow-up questions below:

> On Jul 1, 2015, at 6:03 PM, Thomas Fox <[hidden email]> wrote:
>
> Helge Weissig wrote:
>
>> I forgot about the HTML filter this group uses… pardon the crappy formatting.
>> Here is a text-only version:
>>
>> Our code base is heavily invested in some of the functionality removed from torque 4.0
>> and I was wondering if any one had some advice on a migration path:
>>
>> 1. We make extensive use of village records obtained via
>>         BasePeer.doSelect(Criteria), for example.
>>         Those results could probably be considered a view of the data
>>         and I am wondering if that would be the correct/recommended/best approach.
>
> If you are using this to read data only, have a look at the RecordMapper functionality. A record mapper maps DB Columns to an object. You can use different record mappers for the same table (e.g. it can be passed in through SomePeer.doSelect(Criteria, RecordMapper<TT>), so have different object representations. For a generic village-record-like representation, see org.apache.torque.om.mapper.ObjectListMapper (personally I do not like this representation, put perhaps it serves your needs). Views (now supported !) are also a more db-centric option.

It looks like RecordMapper may ultimately be the best way to go although I still like the idea of using views because it would simplify the assembly of criteria in our code and allow for some generalization as well.


>> 2. We have added our own caching implementation
>>         (configurable via the schema definition) to the Object.vm
>>         and Peer.vm templates. I have read through the new generator documentation
>>         and the customization part of the OR Mapping Reference
>>         (https://db.apache.org/torque/torque-4.0/documentation/orm-reference/customizing-generation.html)
>>         but I don’t seem to be able to find the relevant information
>>         to put it all together. Basically, I would like to override one or more templates.
>>         I think I know how to specify that in an outlet, but do I also need the control configuration
>>         in the conf directory? The maven plugin (we use maven) has configuration parameters
>>         for overrideConfigDir and overrideConfigPackage… do I need to set those?
>
> Have you checked the code-gen tutorial (http://db.apache.org/torque/torque-4.0/documentation/tutorial/orm/index.html) for a basic understanding ?
> Also, there is an example in the torque test project (https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/src/main/torque-gen) which basically does what you want, i.e. overrides the Peer template (adding @SuppressWarnings annotations). The torque generation config which invokes this can be found in https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/pom.xml lines 183ff.
> To be more specific:
> Yes, you need at least an empty control configuration.
> You need to set either overrideConfigDir or overrideConfigPackage, depending whether the additional templates can be found in the file sytsem or in the classpath.
> Please ask again if you cannot get it working. Although more complicated :-(, the new generator is much more extensible than the old one. In the current trunk, you can even use groovy templates :-)

OK, I am getting somewhere, my outlets and templates are now found, alas I am having trouble achieving the desired results. For one, I am not able to get the example at the bottom of http://db.apache.org/torque/torque-4.0/documentation/codegen-reference/configuration.html <http://db.apache.org/torque/torque-4.0/documentation/codegen-reference/configuration.html> to work. I get an error about a missing outlet tag. I think if I could actually override just a specific mergepoint, that would get me 90% of the way. However, we also modified the Bean.vm template to allow us to continue using fields like “modified” or “new” in our DB schema. Those attributes and their corresponding getters and setters are now in the template bean/base/baseBean.vm which is referenced from the bean.xml outlet definition. If I simply bring over that definition, the generator seems to expect to also find all other templates contained in bean.xml to be in my code base. If I remove all mergepoints and other outlets and simply keep the empty outlet definition for bean/base/baseBean.vm, my template is used with a lot of warning messages about missing mergepoint definitions and in the end, the generated beans contain only the above mentioned attributes and their getters and setters and nothing else. Putting it more generally, how can I override parts of a template that also contains a lot of mergepoints?

>> 3. This is more of a maven question, maybe, alas I have the feeling
>>         it isn’t actually possible to implement through it, but is there a way
>>         to skip the code generation steps if the schema sources have not changed?
>
> Sorry, this is not any more possible. The problem is to reliably find out whether the file has changed (e.g. some OS do not change the change date of a file when it is copied). I have seen more than one situation where schema changes did not get picked up because of this option, so my personal recommendation is not to rely on such a mechanism. Also, mvn generate is typically not executed often while developing, so long generation times should not slow down development too much.
> However, if you still want to use it, you can patch the Mojo class in the Torque maven plugin, this should not be too difficult (please share if you choose to do it).

The documentation has the plugin executions happen during the generate-sources phase, which is part of the standard build life-cycle and is executed whenever I run a ‘mvn compile’ for example. I understand the limitations you mention though and will dig into other plugins to see what they do. It may also just be handy to have an option to skip in the plugin, the patch for which I am happy to contribute when I have it.

>
>> 4. I know how to configure logging for the generator, but the switch between loglevel
>>         WARN and INFO is quite severe. Is there a way to log at INFO level but only to a file,
>>         not to the console? We use log4j throughout our project.
>
> Yes, check the log4j configuration documentation. Basically you need to override the log4j configuration file and use two appenders, one file appender and one console appender, with different log levels. The log4j configuration needs to be in the classpath /org/apache/torque/generator/log4j.properties and it must have a higher classpath priority than the torque-generator.jar (I am currently not sure how to achieve this using maven, the last resort is patching the generator)

OK, looking at the code in the generator, this is a little kludgy to achieve via maven itself, at least as far as I can determine. Basically, one would have to package the log4j.properties file under the path /org/apache/torque/generator in a jar file that then would be listed as a dependency for the torque maven plugin. Alternatively, a patch in the generator could check if a logger has already been configured (e.g. via the -Dlog4j.properties system variable) and just use that one if it is the case. I tested both methods and they work. Personally I prefer the second approach and I am happy to contribute the patch if you let me know how.

Thanks again for your help!!

cheers,
h.



> Good luck!
>
>   Thomas
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3.3.1 to 4.0 migration

Helge Weissig
Just a quick follow-up to report on progress on the template conversions. I managed to get the single mergepoint override working by changing the order of outlets and mergepoints. As for the bean classes, I was able to just use the baseBean.vm template and it’s outlet directly from the distribution.

Let me know if you are still interested in the two patches I mentioned below.

cheers,
h.


> On Jul 2, 2015, at 5:04 PM, Helge Weissig <[hidden email]> wrote:
>
> Thanks Thomas! Some comments and follow-up questions below:
>
>> On Jul 1, 2015, at 6:03 PM, Thomas Fox <[hidden email]> wrote:
>>
>> Helge Weissig wrote:
>>
>>> I forgot about the HTML filter this group uses… pardon the crappy formatting.
>>> Here is a text-only version:
>>>
>>> Our code base is heavily invested in some of the functionality removed from torque 4.0
>>> and I was wondering if any one had some advice on a migration path:
>>>
>>> 1. We make extensive use of village records obtained via
>>>        BasePeer.doSelect(Criteria), for example.
>>>        Those results could probably be considered a view of the data
>>>        and I am wondering if that would be the correct/recommended/best approach.
>>
>> If you are using this to read data only, have a look at the RecordMapper functionality. A record mapper maps DB Columns to an object. You can use different record mappers for the same table (e.g. it can be passed in through SomePeer.doSelect(Criteria, RecordMapper<TT>), so have different object representations. For a generic village-record-like representation, see org.apache.torque.om.mapper.ObjectListMapper (personally I do not like this representation, put perhaps it serves your needs). Views (now supported !) are also a more db-centric option.
>
> It looks like RecordMapper may ultimately be the best way to go although I still like the idea of using views because it would simplify the assembly of criteria in our code and allow for some generalization as well.
>
>
>>> 2. We have added our own caching implementation
>>>        (configurable via the schema definition) to the Object.vm
>>>        and Peer.vm templates. I have read through the new generator documentation
>>>        and the customization part of the OR Mapping Reference
>>>        (https://db.apache.org/torque/torque-4.0/documentation/orm-reference/customizing-generation.html)
>>>        but I don’t seem to be able to find the relevant information
>>>        to put it all together. Basically, I would like to override one or more templates.
>>>        I think I know how to specify that in an outlet, but do I also need the control configuration
>>>        in the conf directory? The maven plugin (we use maven) has configuration parameters
>>>        for overrideConfigDir and overrideConfigPackage… do I need to set those?
>>
>> Have you checked the code-gen tutorial (http://db.apache.org/torque/torque-4.0/documentation/tutorial/orm/index.html) for a basic understanding ?
>> Also, there is an example in the torque test project (https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/src/main/torque-gen) which basically does what you want, i.e. overrides the Peer template (adding @SuppressWarnings annotations). The torque generation config which invokes this can be found in https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/pom.xml lines 183ff.
>> To be more specific:
>> Yes, you need at least an empty control configuration.
>> You need to set either overrideConfigDir or overrideConfigPackage, depending whether the additional templates can be found in the file sytsem or in the classpath.
>> Please ask again if you cannot get it working. Although more complicated :-(, the new generator is much more extensible than the old one. In the current trunk, you can even use groovy templates :-)
>
> OK, I am getting somewhere, my outlets and templates are now found, alas I am having trouble achieving the desired results. For one, I am not able to get the example at the bottom of http://db.apache.org/torque/torque-4.0/documentation/codegen-reference/configuration.html <http://db.apache.org/torque/torque-4.0/documentation/codegen-reference/configuration.html> <http://db.apache.org/torque/torque-4.0/documentation/codegen-reference/configuration.html <http://db.apache.org/torque/torque-4.0/documentation/codegen-reference/configuration.html>> to work. I get an error about a missing outlet tag. I think if I could actually override just a specific mergepoint, that would get me 90% of the way. However, we also modified the Bean.vm template to allow us to continue using fields like “modified” or “new” in our DB schema. Those attributes and their corresponding getters and setters are now in the template bean/base/baseBean.vm which is referenced from the bean.xml outlet definition. If I simply bring over that definition, the generator seems to expect to also find all other templates contained in bean.xml to be in my code base. If I remove all mergepoints and other outlets and simply keep the empty outlet definition for bean/base/baseBean.vm, my template is used with a lot of warning messages about missing mergepoint definitions and in the end, the generated beans contain only the above mentioned attributes and their getters and setters and nothing else. Putting it more generally, how can I override parts of a template that also contains a lot of mergepoints?
>
>>> 3. This is more of a maven question, maybe, alas I have the feeling
>>>        it isn’t actually possible to implement through it, but is there a way
>>>        to skip the code generation steps if the schema sources have not changed?
>>
>> Sorry, this is not any more possible. The problem is to reliably find out whether the file has changed (e.g. some OS do not change the change date of a file when it is copied). I have seen more than one situation where schema changes did not get picked up because of this option, so my personal recommendation is not to rely on such a mechanism. Also, mvn generate is typically not executed often while developing, so long generation times should not slow down development too much.
>> However, if you still want to use it, you can patch the Mojo class in the Torque maven plugin, this should not be too difficult (please share if you choose to do it).
>
> The documentation has the plugin executions happen during the generate-sources phase, which is part of the standard build life-cycle and is executed whenever I run a ‘mvn compile’ for example. I understand the limitations you mention though and will dig into other plugins to see what they do. It may also just be handy to have an option to skip in the plugin, the patch for which I am happy to contribute when I have it.
>
>>
>>> 4. I know how to configure logging for the generator, but the switch between loglevel
>>>        WARN and INFO is quite severe. Is there a way to log at INFO level but only to a file,
>>>        not to the console? We use log4j throughout our project.
>>
>> Yes, check the log4j configuration documentation. Basically you need to override the log4j configuration file and use two appenders, one file appender and one console appender, with different log levels. The log4j configuration needs to be in the classpath /org/apache/torque/generator/log4j.properties and it must have a higher classpath priority than the torque-generator.jar (I am currently not sure how to achieve this using maven, the last resort is patching the generator)
>
> OK, looking at the code in the generator, this is a little kludgy to achieve via maven itself, at least as far as I can determine. Basically, one would have to package the log4j.properties file under the path /org/apache/torque/generator in a jar file that then would be listed as a dependency for the torque maven plugin. Alternatively, a patch in the generator could check if a logger has already been configured (e.g. via the -Dlog4j.properties system variable) and just use that one if it is the case. I tested both methods and they work. Personally I prefer the second approach and I am happy to contribute the patch if you let me know how.
>
> Thanks again for your help!!
>
> cheers,
> h.
>
>
>
>> Good luck!
>>
>>  Thomas
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
>> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3.3.1 to 4.0 migration

Thomas Fox
Hi Helge

> Helge Weissig wrote:

> Just a quick follow-up to report on progress on the template conversions.
> I managed to get the single mergepoint override working by changing the order of outlets and mergepoints.
> As for the bean classes, I was able to just use the baseBean.vm template and it’s outlet directly from the distribution.
>
> Let me know if you are still interested in the two patches I mentioned below.

Sorry not to answer sooner. There is not much time nowadays for me and Torque.
I'm glad you got it finally working. Are there still open questions?
Yes, I'm still interested in the two patches. Please create jira issues.

    Thanks,

      Thomas

> cheers,
> h.


> On Jul 2, 2015, at 5:04 PM, Helge Weissig <[hidden email]> wrote:
>
> Thanks Thomas! Some comments and follow-up questions below:
>
>> On Jul 1, 2015, at 6:03 PM, Thomas Fox <[hidden email]> wrote:
>>
>> Helge Weissig wrote:
>>
>>> I forgot about the HTML filter this group uses… pardon the crappy formatting.
>>> Here is a text-only version:
>>>
>>> Our code base is heavily invested in some of the functionality removed from torque 4.0
>>> and I was wondering if any one had some advice on a migration path:
>>>
>>> 1. We make extensive use of village records obtained via
>>>        BasePeer.doSelect(Criteria), for example.
>>>        Those results could probably be considered a view of the data
>>>        and I am wondering if that would be the correct/recommended/best approach.
>>
>> If you are using this to read data only, have a look at the RecordMapper functionality. A record mapper maps DB Columns to an object. You can use different record mappers for the same table (e.g. it can be passed in through SomePeer.doSelect(Criteria, RecordMapper<TT>), so have different object representations. For a generic village-record-like representation, see org.apache.torque.om.mapper.ObjectListMapper (personally I do not like this representation, put perhaps it serves your needs). Views (now supported !) are also a more db-centric option.
>
> It looks like RecordMapper may ultimately be the best way to go although I still like the idea of using views because it would simplify the assembly of criteria in our code and allow for some generalization as well.
>
>
>>> 2. We have added our own caching implementation
>>>        (configurable via the schema definition) to the Object.vm
>>>        and Peer.vm templates. I have read through the new generator documentation
>>>        and the customization part of the OR Mapping Reference
>>>        (https://db.apache.org/torque/torque-4.0/documentation/orm-reference/customizing-generation.html)
>>>        but I don’t seem to be able to find the relevant information
>>>        to put it all together. Basically, I would like to override one or more templates.
>>>        I think I know how to specify that in an outlet, but do I also need the control configuration
>>>        in the conf directory? The maven plugin (we use maven) has configuration parameters
>>>        for overrideConfigDir and overrideConfigPackage… do I need to set those?
>>
>> Have you checked the code-gen tutorial (http://db.apache.org/torque/torque-4.0/documentation/tutorial/orm/index.html) for a basic understanding ?
>> Also, there is an example in the torque test project (https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/src/main/torque-gen) which basically does what you want, i.e. overrides the Peer template (adding @SuppressWarnings annotations). The torque generation config which invokes this can be found in https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/pom.xml lines 183ff.
>> To be more specific:
>> Yes, you need at least an empty control configuration.
>> You need to set either overrideConfigDir or overrideConfigPackage, depending whether the additional templates can be found in the file sytsem or in the classpath.
>> Please ask again if you cannot get it working. Although more complicated :-(, the new generator is much more extensible than the old one. In the current trunk, you can even use groovy templates :-)
>
> OK, I am getting somewhere, my outlets and templates are now found, alas I am having trouble achieving the desired results. For one, I am not able to get the example at the bottom of http://db.apache.org/torque/torque-4.0/documentation/codegen-reference/configuration.html <http://db.apache.org/torque/torque-4.0/documentation/codegen-reference/configuration.html> <http://db.apache.org/torque/torque-4.0/documentation/codegen-reference/configuration.html <http://db.apache.org/torque/torque-4.0/documentation/codegen-reference/configuration.html>> to work. I get an error about a missing outlet tag. I think if I could actually override just a specific mergepoint, that would get me 90% of the way. However, we also modified the Bean.vm template to allow us to continue using fields like “modified” or “new” in our DB schema. Those attributes and their corresponding getters and setters are now in the template bean/base/baseBean.vm which is referenced from the bean.xml outlet definition. If I simply bring over that definition, the generator seems to expect to also find all other templates contained in bean.xml to be in my code base. If I remove all mergepoints and other outlets and simply keep the empty outlet definition for bean/base/baseBean.vm, my template is used with a lot of warning messages about missing mergepoint definitions and in the end, the generated beans contain only the above mentioned attributes and their getters and setters and nothing else. Putting it more generally, how can I override parts of a template that also contains a lot of mergepoints?
>
>>> 3. This is more of a maven question, maybe, alas I have the feeling
>>>        it isn’t actually possible to implement through it, but is there a way
>>>        to skip the code generation steps if the schema sources have not changed?
>>
>> Sorry, this is not any more possible. The problem is to reliably find out whether the file has changed (e.g. some OS do not change the change date of a file when it is copied). I have seen more than one situation where schema changes did not get picked up because of this option, so my personal recommendation is not to rely on such a mechanism. Also, mvn generate is typically not executed often while developing, so long generation times should not slow down development too much.
>> However, if you still want to use it, you can patch the Mojo class in the Torque maven plugin, this should not be too difficult (please share if you choose to do it).
>
> The documentation has the plugin executions happen during the generate-sources phase, which is part of the standard build life-cycle and is executed whenever I run a ‘mvn compile’ for example. I understand the limitations you mention though and will dig into other plugins to see what they do. It may also just be handy to have an option to skip in the plugin, the patch for which I am happy to contribute when I have it.
>
>>
>>> 4. I know how to configure logging for the generator, but the switch between loglevel
>>>        WARN and INFO is quite severe. Is there a way to log at INFO level but only to a file,
>>>        not to the console? We use log4j throughout our project.
>>
>> Yes, check the log4j configuration documentation. Basically you need to override the log4j configuration file and use two appenders, one file appender and one console appender, with different log levels. The log4j configuration needs to be in the classpath /org/apache/torque/generator/log4j.properties and it must have a higher classpath priority than the torque-generator.jar (I am currently not sure how to achieve this using maven, the last resort is patching the generator)
>
> OK, looking at the code in the generator, this is a little kludgy to achieve via maven itself, at least as far as I can determine. Basically, one would have to package the log4j.properties file under the path /org/apache/torque/generator in a jar file that then would be listed as a dependency for the torque maven plugin. Alternatively, a patch in the generator could check if a logger has already been configured (e.g. via the -Dlog4j.properties system variable) and just use that one if it is the case. I tested both methods and they work. Personally I prefer the second approach and I am happy to contribute the patch if you let me know how.
>
> Thanks again for your help!!
>
> cheers,
> h.
>
>
>
>> Good luck!
>>
>>  Thomas
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
>> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>

---------------------------------------------------------------------
To unsubscribe, e-mail: [hidden email]
For additional commands, e-mail: [hidden email]

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 3.3.1 to 4.0 migration

Helge Weissig
No worries! Thanks again for your help earlier.

I have submitted two patches at https://issues.apache.org/jira/browse/TORQUE-337 <https://issues.apache.org/jira/browse/TORQUE-337> and https://issues.apache.org/jira/browse/TORQUE-338 <https://issues.apache.org/jira/browse/TORQUE-338>. The first adds the ability to easily override the logging configuration for the Generator and the second fixes the implementation of the undocumented (hint! hint! ;) ) configuration setting runOnlyOnSourceChange, which already provides for a way to skip the re-generation of sources when the schema files have not changed.

Additionally, I submitted https://issues.apache.org/jira/browse/TORQUE-339 <https://issues.apache.org/jira/browse/TORQUE-339>, which provides a patch for the generator’s configuration documentation.

hth,
h.

> On Jul 7, 2015, at 6:28 PM, Thomas Fox <[hidden email]> wrote:
>
> Hi Helge
>
>> Helge Weissig wrote:
>
>> Just a quick follow-up to report on progress on the template conversions.
>> I managed to get the single mergepoint override working by changing the order of outlets and mergepoints.
>> As for the bean classes, I was able to just use the baseBean.vm template and it’s outlet directly from the distribution.
>>
>> Let me know if you are still interested in the two patches I mentioned below.
>
> Sorry not to answer sooner. There is not much time nowadays for me and Torque.
> I'm glad you got it finally working. Are there still open questions?
> Yes, I'm still interested in the two patches. Please create jira issues.
>
>    Thanks,
>
>      Thomas
>
>> cheers,
>> h.
>
>
>> On Jul 2, 2015, at 5:04 PM, Helge Weissig <[hidden email]> wrote:
>>
>> Thanks Thomas! Some comments and follow-up questions below:
>>
>>> On Jul 1, 2015, at 6:03 PM, Thomas Fox <[hidden email]> wrote:
>>>
>>> Helge Weissig wrote:
>>>
>>>> I forgot about the HTML filter this group uses… pardon the crappy formatting.
>>>> Here is a text-only version:
>>>>
>>>> Our code base is heavily invested in some of the functionality removed from torque 4.0
>>>> and I was wondering if any one had some advice on a migration path:
>>>>
>>>> 1. We make extensive use of village records obtained via
>>>>       BasePeer.doSelect(Criteria), for example.
>>>>       Those results could probably be considered a view of the data
>>>>       and I am wondering if that would be the correct/recommended/best approach.
>>>
>>> If you are using this to read data only, have a look at the RecordMapper functionality. A record mapper maps DB Columns to an object. You can use different record mappers for the same table (e.g. it can be passed in through SomePeer.doSelect(Criteria, RecordMapper<TT>), so have different object representations. For a generic village-record-like representation, see org.apache.torque.om.mapper.ObjectListMapper (personally I do not like this representation, put perhaps it serves your needs). Views (now supported !) are also a more db-centric option.
>>
>> It looks like RecordMapper may ultimately be the best way to go although I still like the idea of using views because it would simplify the assembly of criteria in our code and allow for some generalization as well.
>>
>>
>>>> 2. We have added our own caching implementation
>>>>       (configurable via the schema definition) to the Object.vm
>>>>       and Peer.vm templates. I have read through the new generator documentation
>>>>       and the customization part of the OR Mapping Reference
>>>>       (https://db.apache.org/torque/torque-4.0/documentation/orm-reference/customizing-generation.html)
>>>>       but I don’t seem to be able to find the relevant information
>>>>       to put it all together. Basically, I would like to override one or more templates.
>>>>       I think I know how to specify that in an outlet, but do I also need the control configuration
>>>>       in the conf directory? The maven plugin (we use maven) has configuration parameters
>>>>       for overrideConfigDir and overrideConfigPackage… do I need to set those?
>>>
>>> Have you checked the code-gen tutorial (http://db.apache.org/torque/torque-4.0/documentation/tutorial/orm/index.html) for a basic understanding ?
>>> Also, there is an example in the torque test project (https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/src/main/torque-gen) which basically does what you want, i.e. overrides the Peer template (adding @SuppressWarnings annotations). The torque generation config which invokes this can be found in https://svn.apache.org/repos/asf/db/torque/torque4/trunk/torque-test/pom.xml lines 183ff.
>>> To be more specific:
>>> Yes, you need at least an empty control configuration.
>>> You need to set either overrideConfigDir or overrideConfigPackage, depending whether the additional templates can be found in the file sytsem or in the classpath.
>>> Please ask again if you cannot get it working. Although more complicated :-(, the new generator is much more extensible than the old one. In the current trunk, you can even use groovy templates :-)
>>
>> OK, I am getting somewhere, my outlets and templates are now found, alas I am having trouble achieving the desired results. For one, I am not able to get the example at the bottom of http://db.apache.org/torque/torque-4.0/documentation/codegen-reference/configuration.html <http://db.apache.org/torque/torque-4.0/documentation/codegen-reference/configuration.html> <http://db.apache.org/torque/torque-4.0/documentation/codegen-reference/configuration.html <http://db.apache.org/torque/torque-4.0/documentation/codegen-reference/configuration.html>> to work. I get an error about a missing outlet tag. I think if I could actually override just a specific mergepoint, that would get me 90% of the way. However, we also modified the Bean.vm template to allow us to continue using fields like “modified” or “new” in our DB schema. Those attributes and their corresponding getters and setters are now in the template bean/base/baseBean.vm which is referenced from the bean.xml outlet definition. If I simply bring over that definition, the generator seems to expect to also find all other templates contained in bean.xml to be in my code base. If I remove all mergepoints and other outlets and simply keep the empty outlet definition for bean/base/baseBean.vm, my template is used with a lot of warning messages about missing mergepoint definitions and in the end, the generated beans contain only the above mentioned attributes and their getters and setters and nothing else. Putting it more generally, how can I override parts of a template that also contains a lot of mergepoints?
>>
>>>> 3. This is more of a maven question, maybe, alas I have the feeling
>>>>       it isn’t actually possible to implement through it, but is there a way
>>>>       to skip the code generation steps if the schema sources have not changed?
>>>
>>> Sorry, this is not any more possible. The problem is to reliably find out whether the file has changed (e.g. some OS do not change the change date of a file when it is copied). I have seen more than one situation where schema changes did not get picked up because of this option, so my personal recommendation is not to rely on such a mechanism. Also, mvn generate is typically not executed often while developing, so long generation times should not slow down development too much.
>>> However, if you still want to use it, you can patch the Mojo class in the Torque maven plugin, this should not be too difficult (please share if you choose to do it).
>>
>> The documentation has the plugin executions happen during the generate-sources phase, which is part of the standard build life-cycle and is executed whenever I run a ‘mvn compile’ for example. I understand the limitations you mention though and will dig into other plugins to see what they do. It may also just be handy to have an option to skip in the plugin, the patch for which I am happy to contribute when I have it.
>>
>>>
>>>> 4. I know how to configure logging for the generator, but the switch between loglevel
>>>>       WARN and INFO is quite severe. Is there a way to log at INFO level but only to a file,
>>>>       not to the console? We use log4j throughout our project.
>>>
>>> Yes, check the log4j configuration documentation. Basically you need to override the log4j configuration file and use two appenders, one file appender and one console appender, with different log levels. The log4j configuration needs to be in the classpath /org/apache/torque/generator/log4j.properties and it must have a higher classpath priority than the torque-generator.jar (I am currently not sure how to achieve this using maven, the last resort is patching the generator)
>>
>> OK, looking at the code in the generator, this is a little kludgy to achieve via maven itself, at least as far as I can determine. Basically, one would have to package the log4j.properties file under the path /org/apache/torque/generator in a jar file that then would be listed as a dependency for the torque maven plugin. Alternatively, a patch in the generator could check if a logger has already been configured (e.g. via the -Dlog4j.properties system variable) and just use that one if it is the case. I tested both methods and they work. Personally I prefer the second approach and I am happy to contribute the patch if you let me know how.
>>
>> Thanks again for your help!!
>>
>> cheers,
>> h.
>>
>>
>>
>>> Good luck!
>>>
>>> Thomas
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [hidden email] <mailto:[hidden email]>
>>> For additional commands, e-mail: [hidden email] <mailto:[hidden email]>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [hidden email]
> For additional commands, e-mail: [hidden email]
>

Loading...