Breaking changes in 3.0?
Walter wrote:
I am actually using deeply custom llblgen templates, and writing advanced pluggins to the llblgen designer. And by the way I must say I am amazed to see how flexible the system is and how far customization and extension can be brought. Congratulations! And thanks!
Thank you 
Just one minus point though: the template studio could be brought forward: writing advanced templates can be cumbersome. Adapting existing templates is OK but writting new complex templates is hard. I know there is debugging support, however writing support could be better with better in intellisence, more advanced syntax coloring for really beeing able to distinguish between output text and processing code. Anything in this direction in version 3.0.
The template editor will be merged into v3's designer. We tried to add intellisense but it's actually pretty cumbersome, the main point is that to be able to provide intellisense (other than the pretty meaningless TDL intellisense there is now), the template code has to be known in the bigger picture, i.e. with all the templates. For the output code that's therefore out of the question. For the template code, i.e. .lpt code, it's also impossible, because the template code is also part of a bigger piece of code. We therefore couldn't leverage the intellisense of the edit control, as it can't see valid code when you edit the template.
We defined the background color of the template code to be grey (whitesmoke actually). Do you see a difference in background color? I sometimes noticed on some machines that the background wasn't set to grey (so it was white as well)
I know you are working on version 3.0 of llbgen and would like to know if they are major breaking changes I should be aware of as far as designer pluggins and templates/template studio are concerned? Maybe some features/aspects I should not use?? I would indeed be worried I have to do all the work again when version 3.0 is available.
For the templates, if you're using TDL, there are a couple of statements which became obsolete, and some new ones are added. For .lpt templates, there are many changes, as the object model it talks to (the 'project') is completely different. Therefore, .lpt templates will likely have a problematic migration process. However, we plan to ship the asp.net gui templates with v3.0 as an example, so you can see how things are consumed in v3, and check whether you have to do much migration. If you grouped code together in methods in your templates, it's not going to be a big problem per se.
Plugins same story: the object model is completely different. The plugin interface is still the same, but how an entity looks like etc. is not.
Though I think with the extra features we added in v3, it might be you will drop a couple of templates / plugins as they're no longer needed as well. For example, we added custom setting support, so you can define, for a set of additional templates for example, setting definitions which show up as settings under the project properties for that element, e.g. you define a setting 'GenerateAsComboBox' boolean, which is defined for collection navigators. You can define defaults (which control all navigators of that type) and you can override that value per element in the designer. These setting definitions are simple xml files, so you can easier control what to generate in the designer, and obtain the values for these settings easily in template code.
Walter wrote:
The template editor will be merged into v3's designer. We tried to add intellisense but it's actually pretty cumbersome, the main point is that to be able to provide intellisense (other than the pretty meaningless TDL intellisense there is now), the template code has to be known in the bigger picture, i.e. with all the templates. For the output code that's therefore out of the question. or the template code, i.e. .lpt code, it's also impossible, because the template code is also part of a bigger piece of code. We therefore couldn't leverage the intellisense of the edit control, as it can't see valid code when you edit the template.
Yes I understand that including proper intellisense in templates is a big deal... However that would be a really big bonus. And for reallistic dayly usage of such a tool in a full model driven approach it would be mandatory. I understand you need the full picture of generated code to provide full intellisense. However wouldn't it be possible at least to provide intellisense for classes that are know from assembly inclusion?
The editor control has build-in .net intellisense support. To enable that, one has to provide it with the code the edited text is part of. The problem is that that code is only known if it's been processed, so for every edit you do, the template has to be processed, (which means, the template has to be parsed and generated as C# code which writes output etc.) and that has to be fed to the editor's interpreter so intellisense can be provided. In theory, this is doable, but in practise this might be less optimal, plus we didn't want to invest a lot of time into this yet as other things have higher priority. There's another issue, which is with methods in other templates: you can call them, but when you're editing the templates, it's unknown which templates are in scope, as that's only known at generation time (due to the bindings). To process that in real time is also adding to the performance dip so, although it's doable, it will be time consuming to write it and performance is not going to be optimal.
(edit). I looked up why it's not added in the design document, and I totally forgot about that: the editor control we use (SyntaxEdit) has no support for intellisense when multiple languages are used (as we do: the output language combined with the template language)... so even if we want to add it, it's not really working. One could add it using own parsers, but that's a lot more work than re-using what's in the control at this point. They are looking into adding this, but it's not available at the moment, so this was also a reason why we didn't add it.
We defined the background color of the template code to be grey (whitesmoke actually). Do you see a difference in background color? I sometimes noticed on some machines that the background wasn't set to grey (so it was white as well)
Nope, I don't see the smookey background...
Hmm. What OS are you using? If vista/7, what theme?
For the templates, if you're using TDL, there are a couple of statements which became obsolete, and some new ones are added. For .lpt templates, there are many changes, as the object model it talks to (the 'project') is completely different. Therefore, .lpt templates will likely have a problematic migration process. However, we plan to ship the asp.net gui templates with v3.0 as an example, so you can see how things are consumed in v3, and check whether you have to do much migration. If you grouped code together in methods in your templates, it's not going to be a big problem per se.
Plugins same story: the object model is completely different. The plugin interface is still the same, but how an entity looks like etc. is not.
Yes I understand the object model has changed and I am ready to migrate to new object model. However I believe that the existing concepts (entities, relations etc..) will still existe even though maybe accessible in a different way thus making the migration feasible. I was more worried of a dramatic change of the pluggin and templates system.
No, don't worry about that . The template system will stay as it is. What could change in the future is that we break up templates in further smaller snippets, but the system about templateid's, bindings and the like will stay, as it's flexible enough to deal with changes.
We will add other things like filters per task/taskgroup, but these don't affect your templates.
=
Though I think with the extra features we added in v3, it might be you will drop a couple of templates / plugins as they're no longer needed as well. For example, we added custom setting support, so you can define, for a set of additional templates for example, setting definitions which show up as settings under the project properties for that element, e.g. you define a setting 'GenerateAsComboBox' boolean, which is defined for collection navigators. You can define defaults (which control all navigators of that type) and you can override that value per element in the designer. These setting definitions are simple xml files, so you can easier control what to generate in the designer, and obtain the values for these settings easily in template code.
Great stuff the custom settings system! really something that was missing and required custom pluggins. Anything else interresting and new that would guide me to concentrate on real new stuff and not do twice what is done in V3?
Attributes are now definable in the designer, using a system with defaults at project level and overrides at the element level. Not sure if you use these.
We defined the background color of the template code to be grey (whitesmoke actually). Do you see a difference in background color? I sometimes noticed on some machines that the background wasn't set to grey (so it was white as well)
Nope, I don't see the smookey background...
Sorry, my mistake: I actually see the grey background. However it still not is high contrast to really easily see the border between control code and output code. It is really ok for simple templates but for complex ones, the result is not very easy to read+maintain. I guess this is because of how lpt templates work: they are very flexible in that you can do almost everything you want in a simple way (for what mindset is concern). However not the most readable form. I guess this is why you created the TDL templates: much more readable output, however a bit more complex to understand at first and less powerfull for the range of things you can do (unless I missed something: is it possible to extend the TDL is a way it can handle almost anything you would like to do? easy or cumbersome to extend TDL? haven't tried yet since lpt seemed at first to me to be the most compelling and natural form to write templates)
I am trying to use the Llblgen template editor and code generator not only for the ORM part of the story but also as a more generic template editor/code generator in a Model driven (MDSD) approach. For such a use, I realise that I would need a bit more powerfull tools, like for instance a way to navigate more easily the templates (like kind of a visual studio solution explorer), seemless integration to source control, way to control how code is displayed in lpt templates (more highcontrast? way to switch on/off display of ouput code/ control code for quick view of different viewpoint. etc. etc. Maybe something already in V3, let's see ) However I understand that your focus is on the ORM part, not on the code generator as a generic tool...
By the way, I am deeply using Llblgen v2.6 + template editor/code generator and plan to go even deeper with V3. If there is anykind of Alpha testing of V3 for ealy adopters, please let me know: I will be more than happy to participate!
Kind Regards Walter Almeida
Walter wrote:
We defined the background color of the template code to be grey (whitesmoke actually). Do you see a difference in background color? I sometimes noticed on some machines that the background wasn't set to grey (so it was white as well)
Nope, I don't see the smookey background...
Sorry, my mistake: I actually see the grey background. However it still not is high contrast to really easily see the border between control code and output code. It is really ok for simple templates but for complex ones, the result is not very easy to read+maintain. I guess this is because of how lpt templates work: they are very flexible in that you can do almost everything you want in a simple way (for what mindset is concern). However not the most readable form.
You can actually change this. In the 'LanguageDefinitions' folder, there are a couple of xml files. If you write your lpt templates using C# as logical language, open the SD.Logic.CSharp.xml file into a texteditor (e.g. vs.net). At the top, you'll see <Property Key="DefaultBackColor" Value="WhiteSmoke" />. If you change that to e.g. 'Gray', you'll see a big difference. It accepts named colors, so open a form or control in vs.net and check the colors in the property grid (or open preferences in template studio and check there), and fill in the color there. It should work better. "WhiteSmoke" is very light grey/gray, so with high contrast monitors it might not be visible that much.
I guess this is why you created the TDL templates: much more readable output, however a bit more complex to understand at first and less powerfull for the range of things you can do (unless I missed something: is it possible to extend the TDL is a way it can handle almost anything you would like to do? easy or cumbersome to extend TDL? haven't tried yet since lpt seemed at first to me to be the most compelling and natural form to write templates)
TDL was mainly created for two reasons: 1) I couldn't get the lpt style code generation to work. (honestly!) 2) I wanted a fail safe language so I didn't have to debug templates.
Especially 2 is a big advantage of TDL: you don't have to write large pieces of C# into a template which might contain a bug, it's just a simple set of statements. Downside is of course that it's not turing complete, you can't create code with tdl statements for cases where no TDL statement is defined for, at least not easily. It also doesn't allow you to access everything in the project, which also can be a limitation. For our generated code it works OK, though if I had to do it all over again, I wouldn't create TDL I think, but instead use the code that's now in the TDL interpreter as a public lib which was used in the .lpt templates. We thought of replacing the templates in v3 with .lpt templates but as the templates are many megabytes in size, we dropped that idea for now 
I am trying to use the Llblgen template editor and code generator not only for the ORM part of the story but also as a more generic template editor/code generator in a Model driven (MDSD) approach. For such a use, I realise that I would need a bit more powerfull tools, like for instance a way to navigate more easily the templates (like kind of a visual studio solution explorer), seemless integration to source control, way to control how code is displayed in lpt templates (more highcontrast? way to switch on/off display of ouput code/ control code for quick view of different viewpoint. etc. etc. Maybe something already in V3, let's see
Yes, I understand your points, and if our focus was on the editor more, we really would implement them, however as you guessed, the template editor is not our main concern, the editor is there so people can alter templates, adjust things, and not for writing big template sets in them.
navigation through templates is still limited due to the binding system, which is a bit of a pain in that specific area: you need to navigate templates through the bindings.
By the way, I am deeply using Llblgen v2.6 + template editor/code generator and plan to go even deeper with V3. If there is anykind of Alpha testing of V3 for ealy adopters, please let me know: I will be more than happy to participate!
No alpha yet. Tomorrow we close a new milestone and we'll start with a new milestone (2 weeks) which concerns the template editor itself (so you're right on time! ), porting code over to the designer related to the editor, languages etc. Code generation and the like is all already working, the editor control etc. itself is now going to be added. Your points mentioned above will be taken into account, like navigation...
To get that started: our idea was to navigate template files through the binding files (you can also drag/drop files onto the editor from windows explorer btw!) as those are the ones which make a template be available. There you can select multiple files and edit them. Is that too clunky to work with in template studio? Or do you open templates differently?
Hello Frans, Thank you for your detailled feedback. Below my comments.
You can actually change this. In the 'LanguageDefinitions' folder, there are a couple of xml files. If you write your lpt templates using C# as logical language, open the SD.Logic.CSharp.xml file into a texteditor (e.g. vs.net). At the top, you'll see <Property Key="DefaultBackColor" Value="WhiteSmoke" />. If you change that to e.g. 'Gray', you'll see a big difference. It accepts named colors, so open a form or control in vs.net and check the colors in the property grid (or open preferences in template studio and check there), and fill in the color there. It should work better. "WhiteSmoke" is very light grey/gray, so with high contrast monitors it might not be visible that much.
OK, thanks for information, I will try to change the configuration to see what's best for me.
TDL was mainly created for two reasons: 1) I couldn't get the lpt style code generation to work. (honestly!) 2) I wanted a fail safe language so I didn't have to debug templates.
Especially 2 is a big advantage of TDL: you don't have to write large pieces of C# into a template which might contain a bug, it's just a simple set of statements. Downside is of course that it's not turing complete, you can't create code with tdl statements for cases where no TDL statement is defined for, at least not easily. It also doesn't allow you to access everything in the project, which also can be a limitation. For our generated code it works OK, though if I had to do it all over again, I wouldn't create TDL I think, but instead use the code that's now in the TDL interpreter as a public lib which was used in the .lpt templates. We thought of replacing the templates in v3 with .lpt templates but as the templates are many megabytes in size, we dropped that idea for now
Ok, So I understand from what you say, that probably the best and more complete solution for me (in the direction of writing from scratch complex templates) would be not to use TDL templates. And instead to keep using LPT templates, possibly enriched by a library of functions, reusable across lpt templates, to simplify control code in templates and therefore limit the pain of debugging lpt templates.
To get that started: our idea was to navigate template files through the binding files (you can also drag/drop files onto the editor from windows explorer btw!) as those are the ones which make a template be available. There you can select multiple files and edit them. Is that too clunky to work with in template studio? Or do you open templates differently?
For me what is important is to have a way to easily and quickly navigate/open/edit templates. Currently (v2.6) it is a bit cumbersome: you need first to select a binding file, and then can navigate template files defined in that specific binding file. If you want to open a template defined in another binding, you must open this other binding file...
Ideally I would love to have some kind of a tree structure with first level being the binding files and second level being the templates themselves. If this tree structure is available docked on right or left (like vs studio solution explorer) it would then be very easy to navigate binding and templates!
My workaround by now: I have bound the llblgen "additional templates" and "task" folders to a TFS folder and have added to a vs studio solution, solution folders, to be able to navigate more easily the templates and binding + have source control and quick check in an out. I sometime also edit files for vsstudio but don't have syntax highligting and template compilation... not ideal then!
Hope it helps and can give you ideas!! Kind Regards Walter Almeida
Walter wrote:
TDL was mainly created for two reasons: 1) I couldn't get the lpt style code generation to work. (honestly!) 2) I wanted a fail safe language so I didn't have to debug templates.
Especially 2 is a big advantage of TDL: you don't have to write large pieces of C# into a template which might contain a bug, it's just a simple set of statements. Downside is of course that it's not turing complete, you can't create code with tdl statements for cases where no TDL statement is defined for, at least not easily. It also doesn't allow you to access everything in the project, which also can be a limitation. For our generated code it works OK, though if I had to do it all over again, I wouldn't create TDL I think, but instead use the code that's now in the TDL interpreter as a public lib which was used in the .lpt templates. We thought of replacing the templates in v3 with .lpt templates but as the templates are many megabytes in size, we dropped that idea for now
Ok, So I understand from what you say, that probably the best and more complete solution for me (in the direction of writing from scratch complex templates) would be not to use TDL templates. And instead to keep using LPT templates, possibly enriched by a library of functions, reusable across lpt templates, to simplify control code in templates and therefore limit the pain of debugging lpt templates.
Correct.
To get that started: our idea was to navigate template files through the binding files (you can also drag/drop files onto the editor from windows explorer btw!) as those are the ones which make a template be available. There you can select multiple files and edit them. Is that too clunky to work with in template studio? Or do you open templates differently?
For me what is important is to have a way to easily and quickly navigate/open/edit templates. Currently (v2.6) it is a bit cumbersome: you need first to select a binding file, and then can navigate template files defined in that specific binding file. If you want to open a template defined in another binding, you must open this other binding file...
True. The way templates are organized isn't through the filesystem but through bindings. This is done to avoid people editing templates which aren't showing up during the generation of code, for example because they're not bound to an id.
If you organize your own templates in one or two binding files, wouldn't it be as easy as a treeview? (as you don't have to switch to other bindingfiles anyway)
Ideally I would love to have some kind of a tree structure with first level being the binding files and second level being the templates themselves. If this tree structure is available docked on right or left (like vs studio solution explorer) it would then be very easy to navigate binding and templates!
That would be the case, though you lack a lot of info in treenodes: what's the language, the filename, which ID is it bound to... if you organize it around 1 or 2 binding files, it's not that bad, trust me
[Edit] Sorry for empty messages above, due to my mistake: repost because of error page after posting (but posted anyway...) --- I removed them (Walaa)
I trust you in the scenario of regular use of Llblgen and template studio. However I am trying to use the template studio somehow as a development studio for model oriented development. In this perspective, my need is not to use the template studio only for small adjustments to existing templates but really in a day to day development scenario where I would edit and create templates regularly.
So I have many templates, therefore I organized them in more than just one or two binding files. In my scenario, if I use only one or two binding files, it would be very messy to find my way through all the templates and keep it clear and organized... I therefore need support for organization and navigation through templates. and ideally integration with source control systems etc etc. but that's a lot to ask
I understand that this is not the primary use of the template studio, maybe later
A tree view would have been great... I understand that the templates organisation is not based on file system but on binding files. However the information contained in binding files could be used for a display as a tree view. There would be a lack of information in the tree view (language, filename, templateid etc.), however this information could be included in a property window that would be linked to the tree view and current selection. I understand this is not your focus right now. But I still hope it will come in a near future release 
By the way, I am just thinking of something: I understand that the template studio will be merged in the V3 Designer. It is possible to build plugins for the designer... So... Would it be possible to build a plugin that would be the tree view I am dreaming of???
Kind Regards, Walter Almeida