OrmBattle.net and request to be removed???
Joined: 10-Mar-2006
The site he created was just there to promote his own framework. His tests were deliberately setup in such a way that competing frameworks worked badly. For example his linq tests for llblgen pro failed in 71% of the cases. No offence, but that's just not realistic. Debating that with him resulted in him being incredibly stubborn about that what he used as tests was right and my objections were just bullshit.
I decided we shouldnt be part of this scam and asked for removal. His tests are just bullshit: the lamest O/r mapper with the least amount of features wins.
We weren't #1 rated btw, we were almost at the bottom. This is all a marketing scam. Read what Ayende wrote about the same topic on his blog (Nhibernate also failed miserably)
WayneBrantley wrote:
Frans, To correct the gross error on OrmBattle's part of not including LLBLGEN, I wrote the author and asked him to included LLBLGen. This was let us see how you performed against all others and let others see you as an option when selecting something.
I lost a whole weekend over this nonsense, and it makes me mad every time again I am reminded of it, but that's not your fault, I'm happy that you are so committed to our work and a big fan, and we all want to thank you for your great support in this 
Alex Yakunin wrote me back and indicated that YOU asked to have your product REMOVED from the results. To make matters even more questionable - you were #1 rated!
That's a flat-out lie . We asked for removal because the results were very poor, also because the tests were very much tailored towards his framework (which he later removed as well, not sure if it's back, I don't really care). For example, most tests only succeed properly if the framework supports command batching, something we don't support because we have per-entity authorization, auditing etc., so it makes it impossible to implement.
Further it appears there was a detailed discussion between you guys in your forums which has been removed - so we cannot see why.
I removed that because we didn't want them to give them a podium HERE. If you check the forum there and Ayende's blogposts about this, you'll see it's not a fun thing, e.g. entityspaces also asked for removal, after a tough debate.
Finally, he indicates there were "too many factual mistakes from your side" so you wanted it removed!
See, that's what I'm talking about . Makes your blood boil, at least mine, and it's not worth it. I mean, we're a big framework with thousands of users worldwide, and I don't need a silly debate on some guys blog / marketing site because he thinks I'm wrong. All our customers know we're not wrong, thousands and thousands of applications worldwide show every day we're not wrong and didn't make mistakes. It's not worth it to spend alot of time on this.
Wow...just wow. It seems his testing platform and everything about it is open source. So you would be free to see how the tests were performed and make any needed comments/changes to make the tests 'fair'. When 'tricks' were needed to make nHibernate perform faster, they were implemented, etc. Plus it seems that nearly every product is on the list and llblgen should be on there too....
That's not matching reality, he didn't want to make a single change. He debated endlessly with the nhibernate guys that they did it wrong and didn't understand their framework. He didn't want to change the tests because he thought it was realistic which wasn't the case (inserting thousands of rows in bulk is not realistic)
Joined: 10-Mar-2006
I actually have spoken to this guy at length and indicated it appeared to be biased toward his products, etc.
However, nHibernate is back on the list, he made all the workarounds to make their product as fast as possible. The test framework is open source and you can change it, etc. When he first started this project, I think what you said was more true than it is now with nearly every vendor being there except you and subsonic (since it does not do linq).
While a straight no features open pipe data reader is going to be the fastest - that is still a useful figure to benchmark REAL orm's against a firehose feed. So, while you would not use his chart to locate the product with the most GREEN items to buy, you could certainly use it help evaluate it.
I think with the new 'openness' of the current tests it only hurts you to not be included. Your product, your call, my opinion
WayneBrantley wrote:
I actually have spoken to this guy at length and indicated it appeared to be biased toward his products, etc.
However, nHibernate is back on the list, he made all the workarounds to make their product as fast as possible. The test framework is open source and you can change it, etc. When he first started this project, I think what you said was more true than it is now with nearly every vendor being there except you and subsonic (since it does not do linq).
Of course I can change the code, but that's not the point. The point is the scores on that website.
The real thing however is why these scores are on that website and if that's important. Why they're there is because it's marketing. Is it important? no, benchmarks are never relevant. The thing is that benchmarks are lies, because they never ever tell a story, if your situation will be the same as the tests in the benchmark.
While a straight no features open pipe data reader is going to be the fastest - that is still a useful figure to benchmark REAL orm's against a firehose feed. So, while you would not use his chart to locate the product with the most GREEN items to buy, you could certainly use it help evaluate it.
I don't think that's useful. The thing is that if your O/R mapper is up there with a bunch of red and orange boxes, it looks 'lame'. It's in the picture. No-one will read the fine print. That's why I'm not allowing our results there, because everyone can come up with a bunch of tests to make us look bad. Mind you, we're one of the very few frameworks to beat, for frameworks like theirs, as they lost almost all their customers and started over, effectively starting with a clean slate, and that's not really going to work in todays market.
I think with the new 'openness' of the current tests it only hurts you to not be included. Your product, your call, my opinion
I firmly disagree that it hurts us if we're not on there, it will hurt us if we're on there, for the few people who visit that site.
Early next year we'll release v3. It is designed to support all o/r mappers out there, including our own framework (which we still think is superb, feature wise and technology wise). It's not about the framework and the features, as in the end two frameworks will be left: Nhibernate and entity framework. However to USE these frameworks you need tools, rich tools. That's where we come in, with our designer. We will ship our own runtime because we like our framework, but it's not a key asset for the future, as most people will either use nhibernate or EF. We won't drop our own framework as it's feature complete and solid, but it's not a thing to win customers with anymore. That's the **essential **part. The tools surrounding the frameworks are, and we have a superior offering for that. The frameworks... they all can do o/r mapping properly nowadays, one has feature ABC, the other has XYZ.
The tools, those are the important part. We know our framework rocks, the tools rock and will only get better soon with v3, and I don't need a site with skewed benchmarks to try paint a wrong picture about us.
To clarify a bit about what I meant with 'in the end there will be nh and ef' I've tried to explain it a bit below.
An O/R mapper system as we (LLBLGen Pro team) see it is a) a framework and b) tooling to use that framework in the most productive way possible. Tooling depends on framework, and framework depends on tooling.
In the past years, O/R mappers mostly offered just a), the framework and the quality of these frameworks differed a lot with respect to what features were really implemented and if they were solid in every situation (so no weird things in edge cases). To get more users / customers, the O/R mapper vendors and teams tried to show as much features as possible and tried to implement as much features as they could to get the 'best' framework out there and simply offer the best there is. 'Best' being the framework with the longest feature list.
This of course can't continue till oblivion: there's a moment where a framework has all the necessary features and all features added to it are just fluff and not really features to make a person decide to pick that framework over the others.
So frameworks in the end evolve all to a state where they have all the features a user could want and perhaps some more features than others, but that's it. This means that a framework's featureset isn't the way to persuade a customer to pick that framework. Perhaps it's a bit slower in some situations but it makes up for it in other situations etc., that's ok, the user will see that in general all featuresets are equal and offer the things s/he's looking for.
Among O/R mapper vendors / developers we have an understanding what O/R mapper systems do: they do data-access and on top of that they provide entity services, like change tracking, authorization, validation etc. With enough time, a team can develop the data-access part and then all the services you possibly could imagine and more.
There are two teams in the world who have that amount of time: Entity framework team and the NH team. They also don't have the worry that they have to get customers buying their stuff because they're both free, one is even installed on every windows system with .NET 3.5 sp1, and as they don't have to compete in the market, they can spend their time on adding all the services they can possibly imagine.
A commercial o/r mapper framework vendor can't do that: if the vendor spends all the time available on features of the o/r mapper framework and keeps on adding new services and features, it will go belly up. The reason for that is simple: features and services are not a way to get customers anymore and you can't compete with teams who have unlimited time and in the case of EF almost 'unlimited budget' and manpower (I was told 150+ people a few years ago, likely more today, due to the commitment of MS on the framework) and which are both free as well!
So what is important then for a commercial O/R mapper system vendor?
When I started this post, I wrote that there are two things: framework and tooling. There are more things of course, like solid support, reliability, short communication lines with team, quick fixes etc. etc. Ultimately the customer will take these or a subset of these into account when going for a given O/R mapper system. However tooling is very important, together with support it's what a company can offer additionally to a framework.
A company has to make money or it will go out of business. When EF was announced, we realized that ultimately, people would mostly move to that framework because there will be a truckload of books / docs etc. available, courses to take at every corner, articles to read everywhere etc. and it's free and above all: it's from MS and a lot of companies think that's important (luckily not that much anymore as years ago 
). As you can't beat a framework in a feature game, as in the end they will get the features we have, we had to take a different road. We realized we had another big pair of assets: support (this forum, our great support team and the quick way we are able to fix problems) and our designer.
The designer is the key to stay in business, not the framework. We will keep our framework as it allows us to provide a framework and services in a form we think is great, and for the people who think so too, it will be available and updated with every release. Though our designer should support these other frameworks too, as the other frameworks in general have lousy tooling if at all: frameworks can't live without tooling, the tooling needs frameworks.
So what does this all mean for the future (v3 and onwards)? I've made a bulleted list to keep things simple:
- You can keep using our designer and enjoy our support quality
- You can keep using our runtime framework and enjoy the features it has and will get in the future. We love our framework and will not abandone it.
- You can also use our designer and the vast amount of features it has and will get in the future releases for other frameworks like EF, L2S, NH and others, if your work environment requires that (e.g. the developers writing the apps are used to EF or NH)
- You don't have to worry about the 'feature game' which framework is 'best', you can use any framework the designer supports (and our plan is to support them all ). So in short: the feature game is over, it's not relevant anymore
I hope this clarifies it a bit and also gives a bit of insight why it's irrelevant for us to be on a site like ormbattle.net, simply because the site and tests favor the most lamest frameworks. I checked this morning, and the framework with the most green blocks was a framework with hardly any 'entity services', just a bare bones object reader/writer with some code around it. Other things are more important, much more important, and I hope the post above clarified that a bit.
Joined: 10-Mar-2006
Nice post, that helps clarify your plans alot.
Now, quit typing and get back to work so we can get this next version! 
WayneBrantley wrote:
Nice post, that helps clarify your plans alot. Now, quit typing and get back to work so we can get this next version!
Sir, yes sir!
I made a hell of a progress today with model views, hopefully I get it all wrapped up tomorrow so I can post a new video. I hope that satisfies you a bit till the beta is here in january
Great post indeed!
Can't wait to see the V3 I really believe in the quality of your work and I know you will (again) produce high quality software!
And IMHO it is good that you react on the maturity of the market and progression of other ORMs (and especially EF) and adapt your business plan to still be in for a long time.
I have seen that you plan features like DSL/text quick modelling which is great too and is a must have for a fully-fledge model based toolset. That would be going in the direction of M, wouldn't it? Any plan for supporting M in the future? if one day M is getting feature complete?
Keep the hard work Frans!
It wouldn't go in the direction of M nor will we support that. We started with a text-DSL for v3, but it turned out to be impractical, because of the 'rename' problem: text-based DSLs have to be post-processed to something else, like xml, classes etc., but you can't use them as a representation of a live model (so changing a string in the editor live updates the model), unless your editor allows you to edit the live AST tree. As I know intentional software did that with years of work and many smart people, I knew that wasn't the way to go (neither is M, btw).
So I did something else.
The text-input system QuickModel is in, I finished the input parser/executor yesterday. You can type in single-line statements like: Customer.Orders 1n Order.Customer and it will create any elements that aren't there, so you can quickly create a model while typing simple statements.
Joined: 10-Mar-2006
Enough! Quit toying with us already and show us a beta......or another video.
WayneBrantley wrote:
Enough! Quit toying with us already and show us a beta......or another video.
well, as if I heard you!
http://weblogs.asp.net/fbouma/archive/2009/11/25/llblgen-pro-v3-0-model-first-with-quickmodel-and-more.aspx
posted a couple of minutes ago. I think this will wet your appetite
@Walter, please watch the video to see what v3 has for text dsl and modeling.
Models as text is really not that useful, as with models which become bigger, you'll see a lot of definitions and you'll lose overview. Also, you need definitions for entity, mapping and meta-data, so it gets messy quickly when looking at text.
Changing text is, as I said earlier, not possible to be reflected in the model in real time, unless you're editing directly in a tree. MS nor we have that, so text editing is always editing a definition which is parsed afterwards (even if it looks as if it's on the fly). M does this too, which has serious drawbacks. Not in demos, but in real scenarios. I therefore think that what we've created is more flexible.
Joined: 10-Mar-2006
No worries, I am under no delusions that you actually listen to me.
WayneBrantley wrote:
No worries, I am under no delusions that you actually listen to me.
actually, I did listen . You can group entities in the designer, and generate per group a separate project . See? You liked the video btw?
Joined: 10-Mar-2006
WayneBrantley wrote:
Thoughts: 1) Can I somehow apply attributes (data annotations) directly to the model generated and not have to use buddy classes?
Yes. You can define them using macros (so they're target language agnostic) and you can define them on elements like entities, navigators, fields etc. at the project level which are then inherited by the elements you have in your model. For example if you define the attribute Browsable($false) at the single-entity navigator element at the project level, you'll get that attribute defined for every single-entity navigator (e.g. order.Customer). However, at each element level, you can decide to ignore that attribute (so disable it for that element) and add your own. So it's very flexible and easy to use, and you can define any attribute, and with the macro's its also easier, for example by using macro's for the element's name etc.).
2) Can we use our own Enum's as types in the select the type? You said user defined ones are shown there?
yes. This is done through a small xml file in the folder the project file is located in, so importing enum types is very simple and flexible.
3) Need great MVC, RIA and WCF data services support, etc - I suppose that comes with not using LLBLGen framework but another one. I know there are some user contributions on this...
The frameworks should work as-is, but of course MS creates edge cases which need special code. We don't focus on these frameworks though try to add as much extension points as possible to make llblgen pro code usable with for example mvc, wcf etc.
4) Can we easily 'compare' the current model to the database to see what changes need to be made (or that your tool is about to make)?
Every change on the meta-data can be exported as an UPDATE script So it generates DDL SQL for you which updates the database schema with the changes made. You can also create a create DDL SQL script. So for example if you remove a field or add a field to a table, you can generate an UPDATE script which removes just that field or adds just that field to that table. No comparison tools required. Tracks renames as well
Joined: 10-Mar-2006
WayneBrantley wrote:
1) Macros - great, so I can output something like Length($size) and it would update the output to reflect the size of the field in the model? Awesome.
yep
3) Yeah, I hear you, but 'other' frameworks just have wizards that create everything for you and it just works out of the box - ultimate that is what I want to have and start with. Like a model binder for MVC, etc...
I'm not sure I follow you, as EF with WCF for example works as LLBLGen Pro with WCF? (for example)
4) cool - however, generally I just want a summary of the changes, not really to read the update script. so......field x renamed to y, field z removed, foreign key added on blah to blah...
It's just a template based output, so if feedback requests that kind of report, it's easy to create it
Looks like you have hit all my issues squarely on the head! Good job.
Thanks!
Otis wrote:
@Walter, please watch the video to see what v3 has for text dsl and modeling.
Hello Frans, Yes I have seen the demo, really looking promising, great job!
Otis wrote:
Models as text is really not that useful, as with models which become bigger, you'll see a lot of definitions and you'll lose overview. Also, you need definitions for entity, mapping and meta-data, so it gets messy quickly when looking at text.
Changing text is, as I said earlier, not possible to be reflected in the model in real time, unless you're editing directly in a tree. MS nor we have that, so text editing is always editing a definition which is parsed afterwards (even if it looks as if it's on the fly). M does this too, which has serious drawbacks. Not in demos, but in real scenarios. I therefore think that what we've created is more flexible.
I think I see where you getting from. makes sense. Models are more complex than for exemple aspx markup files with multiple references of same elements in several place + complex validation and integrity rules. Quickmodelling and command line approach at least let one specify quickly parts of the model which is good and seems a great approach. One question reagarding the Quick modelling :
Do you think it could be possible to make this quickmodelling system extensible? let say : if I develop a plugin, can I extend the quick modelling to add commands to control parts of my plugin?? would be awesome
Another question that came to my mind with the talk on diffs on database schemas: In real world scenario of using Llblgen in project and integrate in the development lifecycle, it is a good practice to integrate the Llblgen project file in the version control system you're using. No problem with that. Also a good practice to do reviews of changes between two version, before shipping. This is something that you can do easily with source code, especially if you associate source code changes to workitems to allow targeted code review (you can do that with TFS). My question is: do you have a way to reviews changes between two models (not at the database schema level, which is an output, but at the model level)? This would be just great!!! So I can review and validate that between two versions of a model, just the changes I expected are done, no bad surprise in production to have missed something.
Kind Regards Walter
Walter wrote:
Otis wrote:
@Walter, please watch the video to see what v3 has for text dsl and modeling.
Hello Frans, Yes I have seen the demo, really looking promising, great job!
Otis wrote:
Models as text is really not that useful, as with models which become bigger, you'll see a lot of definitions and you'll lose overview. Also, you need definitions for entity, mapping and meta-data, so it gets messy quickly when looking at text.
Changing text is, as I said earlier, not possible to be reflected in the model in real time, unless you're editing directly in a tree. MS nor we have that, so text editing is always editing a definition which is parsed afterwards (even if it looks as if it's on the fly). M does this too, which has serious drawbacks. Not in demos, but in real scenarios. I therefore think that what we've created is more flexible.
I think I see where you getting from. makes sense. Models are more complex than for exemple aspx markup files with multiple references of same elements in several place + complex validation and integrity rules. Quickmodelling and command line approach at least let one specify quickly parts of the model which is good and seems a great approach. One question reagarding the Quick modelling : Do you think it could be possible to make this quickmodelling system extensible? let say : if I develop a plugin, can I extend the quick modelling to add commands to control parts of my plugin?? would be awesome
not at the moment.
Another question that came to my mind with the talk on diffs on database schemas: In real world scenario of using Llblgen in project and integrate in the development lifecycle, it is a good practice to integrate the Llblgen project file in the version control system you're using. No problem with that. Also a good practice to do reviews of changes between two version, before shipping. This is something that you can do easily with source code, especially if you associate source code changes to workitems to allow targeted code review (you can do that with TFS). My question is: do you have a way to reviews changes between two models (not at the database schema level, which is an output, but at the model level)? This would be just great!!! So I can review and validate that between two versions of a model, just the changes I expected are done, no bad surprise in production to have missed something.
It's xml and ordered in such a way that changes to a model are the only changes in the xml, so everything else is at the same spot in the same layout. This makes it easy to spot changes from 1 version to the next.
It's xml and ordered in such a way that changes to a model are the only changes in the xml, so everything else is at the same spot in the same layout. This makes it easy to spot changes from 1 version to the next.
Ok, That's great! It often happens that XML project or output files are using so complex schemas or including so much internal information, or just changing order of elements from one version to the other that making diffs of them is useless, even for small changes (I am thinking of SharePoint XML Exports).
So good news
Yes, I designed the xml format for merging so it's structured in a way that it's easy to merge, spot differences etc. It's also not build like soap xml with reference ids, but with names. It's a big step forward compared to v2.x
Joined: 10-Mar-2006
As an unnamed person above said:
"Enough! Quit toying with us already and show us a beta...... "
I really expected the beta on 1/1.
Quit stalling and let me at it! 
Next week (fingers crossed)