N-Tier development

[Sorry for the re-post.  The formatted code is messing with the overall formatting of this page.]
(Ok, I'm going to try to recapture the post as best as I can, but in less time... sorry it's going to seem a little more rushed than I would like).

(Also, Chris be sure to read carefully... you may find an answer to your question.)

Ok, so here's the “reader's digest version” of what n-Tier development is.  Originally this concept was called “3 tier development”  The 3 tiers are Presentation (your GUI), Data (getting data from the database), and Business (the actual rules associated with your data...)  In time their were wars about whether 3 was too many or not enough.  People made good cases for both ways (I happen to embed much of my business logic in either the Stored Procs or the data later itself as validation rules... I guess some of the validation logic also makes it's way to the font end GUI too).  The idea was that you separate these pieces in an application.  A practical way to do this is to put it into a separate folder in your project or create a new project entirely.

Why do this?
Here's a simple scenario (but it's not hard to find instances in the past if you've done this for any length of time).  Let's say you have a client that has techs in the field that they want to update tickets.  Since the Internet is all the rage they decide that an asp.net web app would fit the bill, so you work hard and create the site from nothing (and you don't use an N-Tier methodology).  A few months later the client comes back and says that the laptops are kind of expensive and they are wondering if you could give them some kind of Phone app... at this point your scratching your head because even though you've already written a big chunk of the code you will probably need to rewrite everything... And you need to be competitive or else you won't get the job. 

With N-Tier, you can easily identify what parts of your app are data-related and yank them right out of the web project and dump them into a smart device app, or (better yet) a web service.

So how do I do it?
What I do (and by no means does this represent an absolute best practice... but this works for me... those more experienced please correct me) is I build what I call “SQL” classes.  Let's take a simple “MOCK” authentication scenario (NOTE: by no means am I suggesting that this is the only code you would want to use for custom Authentication).  So I start out with a simple base class that all my SQL classes derive from (I simply have some standard functions that I use):

 

1Option Explicit On
2Option Strict On
3
4Imports System.data
5Imports System.Data.SqlClient
6
7Public Class sqlBase
8  Protected _Database As {Some Namespace}.Database
9
10  ' Common Field Sizes
11  Protected Const USERID_PARAM_SIZE_IN_CHARS As Integer = 100
12
13  ' Common Parameters (I can more or less templat-ize my stored procs in one place... 
' this one is used everywhere in the app)
14 Protected Shared ParamUser As SqlParameter = _
New SqlParameter("@USER", SqlDbType.VarChar, USERID_PARAM_SIZE_IN_CHARS) 15 16 ' ReturnCode Param 17 Protected Shared ParamReturnCode As SqlParameter =
New SqlParameter("@RETURN_CODE", SqlDbType.Int) 18 19 Public Sub New() 20 _Database = New {someNamespace}.Database 21 ParamReturnCode.Direction = ParameterDirection.ReturnValue 22 End Sub 23 24' With these 2 functions I can clone one of my shared params 25 Protected Function CreateParamFromTemplate(ByRef paramTemplate As SqlParameter, _
ByVal value As Object) As SqlParameter 26 27 Try 28 Dim param1 As SqlParameter =
DirectCast(DirectCast(paramTemplate, ICloneable).Clone(), SqlParameter) 29 param1.Value = value 30 Return param1 31 Catch ex As Exception 32 Throw New DataException("Error in sqlBase:CreateParamFromTemplate", ex)
' I have my own exception class 33 End Try 34 End Function 35 36 Protected Function CreateParamFromTemplate _
(ByRef paramTemplate As SqlParameter) As SqlParameter 37 Try 38 Dim param1 As SqlParameter = _
DirectCast(DirectCast(paramTemplate, ICloneable).Clone(), SqlParameter) 39 Return param1 40 Catch ex As Exception 41 Throw New DataException("Error in sqlBase:CreateParamFromTemplate", ex)
' I have my own exception class (check out the Exception App Block) 42 End Try 43 End Function 44 45End Class

There is a Database class mentioned in this definition; I'll let you dream it up; all it does is we get the connection string.

Alright now we mentioned that we want to do authentication, so a simple Security class would look like this:

6
7Option Explicit On 
8Option Strict On
9
10Imports System.Data.SqlClient
11Imports Microsoft.ApplicationBlocks.Data
12
13Public Class sqlSecurity
14  Inherits sqlBase
15
16' Stored Proc names
17  Private Const AUTH As String = "SP_Auth"
18
19  '  Field Sizes
20  Private Const PASSWORD_PARAM_SIZE_IN_CHARS As Integer = 1000
21
22  ' Parameters 
23  Private Shared ParamPWD As SqlParameter = _
New SqlParameter("@PWD", SqlDbType.VarChar, PASSWORD_PARAM_SIZE_IN_CHARS) 24 25' A simple Authentication function... not meant to illustrate a great example of security) 26 Public Function Authenticate(ByVal UserID As String, ByVal Password As String) As Boolean 27 Dim __DR as SQLDataReader 28 Try 29 __DR = SqlHelper.ExecuteReader( _ 30 _Database.ConnectionString, _ 31 CommandType.StoredProcedure, _ 32 AUTH, _ 33 CreateParamFromTemplate(ParamUser, Left(UserID, USERID_PARAM_SIZE_IN_CHARS)) _ 34 CreateParamFromTemplate(ParamPWD, Left(PWD, PASSWORD_PARAM_SIZE_IN_CHARS) ) 35 return __dr.read() ' There is a record if the user authenticates (otherwise no record returns 36 Catch ex As Exception 37 Throw New SecurityRelatedException("Error in Get User Permissions", ex)
' Once again I have derived my own exceptions (see the MS Exception Block for more info)
38 Finally 39 if Not (__DR is Nothing) andalso __DR.IsClosed = False Then __DR.Close() 40 End Try 41 End Function 42 43End Class

And finally how you would call this

1Dim __SQL as sqlSecurity
2Try
3  __SQL = New sqlSecurity()
4  If __SQL.Authenticate(UserID.Text, Password.Text) Then
5     ' User is logged in do get them into the app
6  Else
7     ' User is not logged in.. show them the door
8  End If
9Catch Ex as Exception
10   ' Do something with the error
11End Try

The important thing is that my web (or win) code does not touch the database at all.  That means that the database code can “live” anywhere (in the app, from another DLL, from a web service, etc.).  I'm much more flexible this way.

I will probably followup on this and explain the code more if necessary.  Let me know via comments if I need to (BTW, Chris, my guess is that you should keep the code in the web service...).





 

Print | posted on Thursday, June 03, 2004 7:46 PM

Feedback

# re: N-Tier development

left by at 6/3/2004 6:15 PM Gravatar

Not bad, that in fact is probably one of the most common models I've seen to date.

<br>

<br>I would like to offer this concept though. Keep the data access as simple as possible and &quot;try&quot; to put as much business logic/rules into the actual business services layer.

<br>

<br>Now, of course, simple projects don't really need good abstraction, they can make do with something simple that just acts as a wrapper for the stored procedures or sql statements. Even sometimes, embedded SQL is far easier than having a stored procedure and all that.

<br>

<br>However, we are thinking n-tier. We are thinking n-tier logically. What if you need to go to a physical n-tier model. You can do a lot of refactoring to rip out the code you need to move and create proxies or wrappers to call it.

<br>

<br>Or, you can build up front a mechanism that will create a single point of maintenance if you ever need to physically distribute your application.

<br>

<br>Important thing to note is that, unless you have real metrics that show you will need to distribute to gain the performance level you require, do not distribute. However, build in such a way that you can distribute if you suspect the need will &quot;eventually&quot; come.

<br>

<br>So, what is the magic pill for making sure that you can distribute if you need to? The factory pattern. Some very simple code can make a huge difference in the flexibility of your application. Here's some code I've been working with lately on doing just this.

<br>&lt;code&gt;

<br>public static Service CreateInstance(Type serviceType)

<br>{

<br> if (serviceType == null)

<br> {

<br> throw new ArgumentNullException(&quot;serviceType&quot;);

<br> }

<br> else if (!serviceType.IsSubclassOf(typeof(Service)))

<br> {

<br> throw new ArgumentException(&quot;The service type is not a subclass of a service.&quot;, &quot;serviceType&quot;);

<br> }

<br>

<br> StarsServicesConfiguration configuration = StarsServicesConfiguration.GetInstance();

<br>

<br> if (configuration.Remoting.Contains(serviceType))

<br> {

<br> return (Service)Activator.GetObject(serviceType, configuration.Remoting[serviceType]);

<br> }

<br> else if (configuration.Proxies.Contains(serviceType))

<br> {

<br> return (Service)Activator.CreateInstance(configuration.Proxies[serviceType], true);

<br> }

<br> else

<br> {

<br> return (Service)Activator.CreateInstance(serviceType, true);

<br> }

<br>}

<br>&lt;/code&gt;

<br>The &quot;Service&quot; class is the base class for all services, you then have a configuration section that allows you to configure a service to use a proxy or a remoting url. So essentially, to create an instance you would use code like this.

<br>&lt;code&gt;

<br>UserService service = (UserService)Wegmans.Stars.Services.Service.CreateInstance(typeof(UserService));

<br>&lt;/code&gt;

<br>It is a very flexible way to configure your business services, or you could apply the same techniques to data access methods if you really had to.

<br>

<br>The idea of having to remote some object and procedure is the norm for distributing within the application. But what if all of a sudden, say your FooService doesn't really need to be synchronous and an asynchronous process that must succeed is required? You could write a wrapper around the service that would hand off all the methods to say an MSMQ queue that a queue listener would pick up and process. Or maybe a web service, or another process.

<br>

<br>Most of us look at design patterns and say that's nice, but how do I get it into my application design. Most of the time, patterns are very handy to have, but RARELY will you see a pattern implemented EXACTLY as it is described.

<br>

<br>I am all for constructors, don't get me wrong, I use them on things that do not need to have this flexibility. But I am finding, where there are critical chunks of code, factories work wonders.

<br>

<br>Another pattern I am found of is a data mapper. Well, in my case I use it as a mechanism to take an IDataReader and create an object from it.

<br>

<br>How do you know which mapper you need for which business entity? Well usually you can name them the same thing, but what I like to do is offer a little more customizability like the service factory method. Just use your imagination...

<br>&lt;code&gt;

<br>// DataMapper.CreateInstance(Type mapperType, params int[] fields);

<br>// DataMapper.CreateInstance(Type mapperType, IDataReader reader, params string[] fields)

<br>DataMapper mapper = DataMapper.CreateInstance(typeof(User), 0, 1, 2, 3, 4);

<br>User user = (User)mapper.CreateFrom(reader);

<br>&lt;/code&gt;

<br>I don't care about what mapper I need to use, I just want one that will do what I want or need.

<br>

<br>Maybe I should post in my blog about this, huh? :) Well I hope you get the idea. The business services should be dumb about the actual data store. The interface to the data store should not even suggest that the actual data is residing on a SQL database.

<br>

<br>I may get some heat, but I think this is an important concept that many developers miss or don't even strive for. I will agree though, that some projects do not warrant this indepth design.

# re: N-Tier development

left by at 6/3/2004 6:36 PM Gravatar

I kind of have some idea what you are talking about Adam...

<br>

<br>Since OOP as an everyday methodology what you are talking about is still something a little ways down the road for me (I'm building classes/objects now which is a change from what I used to do).

<br>

<br>BTW, I think I'm going to do another post tomorrow that is a step back from all this... I think there may be a few users out there that have no clue what we mean when we use terms like &quot;inherit&quot; or &quot;derive&quot;... and even &quot;instance&quot; is a strange word some VB guys.

# re: N-Tier development

left by at 6/3/2004 6:40 PM Gravatar

Great information here even though it doesn't get me exactly what I was looking for. So Jay, you use SPs and just wrap them in your DAL? That's more or less how I have it setup now and may be the way I need to keep it, I was just curious if it would be better overall in the long run to use hard coded SQL or even generate the DAL with an OR mapper. I have some pretty large SPs (a few that use cursors) and I didn't know if duplicating that code into the DAL would be horribly inefficient or if it wouldn't make a big performance difference. While I'm at work tomorrow I will take some time and put together a post on how the application has been designed to function, both for clarity and since it might give some other bloggers application ideas.

# re: N-Tier development

left by at 6/3/2004 6:42 PM Gravatar

Oh yeah Adam.  One thing you mention that really does annoy me.  Stored Procs are not an optional thing in my world.  Too much bad things can happen from a security perspective not using stored procs.  Inline SQL is evil (and I'll write a post about that someday).  

<br>

<br>There are also way too many benefits to using Stored Procs than using inline SQL (I know that there are those who say that it's a way to making to switch to a different databases easy, but you know what... Switching an app from Windows to Unix (correctly) is not a simple deal... why do people think it should be easy to do it with another platform -- like from SQL Server to Oracle!

# re: N-Tier development

left by at 6/3/2004 7:09 PM Gravatar

I would leave the SQL code in SQL.  

<br>

<br>BTW, you remember the DBA who doesn't use comments (she is a really good DBA, btw (despite the &quot;no comments thing&quot;) .

<br>

<br>She would tell you that there has to be a way to do what you're doing without cursors...

<br>

<br>I personally would have to agree to her statement, but the cursor problem si something I'd have to be working on to give you an answer for.

<br>

<br>Another (unrecommended method -- though not as costly performancewise as cursors) would be to use Temp tables (BTW, the same DBA hates temp tables too, but she maintains the database on a really large E-Commerce site., so she lives eats breathes sleeps performance...

<br>

<br>If want more info use the form on the side and email me and I'll give you a little more detail on temp tables (they might be easier to use to replace your cursor(s))...

<br>

# re: N-Tier development

left by at 6/4/2004 5:03 AM Gravatar

Personally, I use stored procedures all the time just because it is a safety net for me. Another level of encapsulation for the actual data access. It helps keep things simple and where they belong.

<br>

<br>When someone says, stored procedures make an application less portable, I usually like to respond with. Your application architecture should really be the supporting key piece for which data storage method you decide to use. The actual implementation of the data access layer should be responsible for this. If it needs to be customizable the provider pattern has been proven to be reliable and easier to manage than generic sql. It also allows a data provider to be optimized as much as possible for the data storage device it represents.

<br>

<br>But, not all databases support nice stored procedures or stored procedures at all. In some cases it needs to be in the data access layer because it cannot be supported at a lower level.

<br>

<br>Some of the techniques I use are definitely evident of a solid evolution of understanding of what designs are good theory and what work. I always write test cases for any theory before I suggest it is a good thing. I would encourage all others to do the same.

# re: N-Tier development

left by at 6/4/2004 5:46 AM Gravatar

Sorry Adam,

<br>

<br>I guess I overreacted on the SP topic.  I've been seeing way too many code generators that produce inline SQL like it's a good idea.

<br>

<br>Chris Wallace seems to be swimming in the midst of all this.  I really want to try to help him out.  He actually a posted a question about it and got some answers that suggest that there is a time to not use them (I disagree... unless you're using something like (God forbid!) MySQL.

# RE: N-Tier development

left by at 6/4/2004 6:05 AM Gravatar

How many tiers you code in is purely situational.

<br>

<br>As far as in-line sql vs stored procedures:

<br>

<br>

<br>In-Line

<br>--------

<br>+ Tends to be easier to write at the time of development

<br>+ Easily portable to different database systems

<br>- Requires a recompile / redeploy on sql changes

<br>

<br>

<br>Stored Procedures

<br>----------------------

<br>+ Can make data layer logic changes without recompile

<br>+ More granular security

<br>+ Better performance for complex sql

<br>- Not as portable to different database systems

<br>

<br>

<br>In my very humble opinion performance is the last reason you should have for using stored procedures.  Really complex sql sure, stored proc will probably outperform in-line sql.  But your basic CRUD statements it will make no difference performance wise.

<br>

<br>Neither technique saves you code, you're either building a sql statement in line or building parameters in line.  So that's a wash.

<br>

<br>So in the end as long as I know that my app is going to stay on whatever type of database server I'm working with, the security needs to be a little more granular, and there are some complex sql statements that exist... I will use stored procs.

<br>

<br>If I need portability and I'm not overly concerned about security then I use in-line.

# re: N-Tier development

left by at 6/4/2004 6:15 AM Gravatar

I usually throw the performance thing out as a quick answer.

<br>

<br>I actually have a Mac OS X Web Project that I built using nothing but open source toole (long story).  When I was looking at database servers, I chose Postgres over MySQL, because Postgres supports stored procs (even though there is still alot of inline SQL in my project).  Re-use of that code was imporant to me (as well as being able to easily locate the SQL).  At the time MySQL also didn't support transactions which was the ultimate deal killer for me.

# re: N-Tier development

left by at 6/4/2004 6:25 AM Gravatar

Here are my arguments for stored procedures....

<br>

<br>+ Data integrity

<br>

<br>With normal crud statements, maybe you want to prevent the INSERT from even happening if the data coming in doesn't the correct criteria.

<br>

<br>For example, maybe a name has to be at least one character long, ok you can check this in code no problem but what if someone by passes your code to do the update in the database.

<br>

<br>Another example is foreign key constraints, I always do lookups to make sure that the keys exist before I do anything to them. By the time I actually perform the DML statement I am 99.999% sure that it is going to succeed.

<br>

<br>+ Maintainability

<br>

<br>With embedded SQL you typically have to either debug to where the full SQL statement is compiled to view it then run it against the database to find out the problem or you need to write the compiled sql statement to a stream so that you can do something with it.

<br>

<br>With stored procedures you can debug each one by some simple test cases. If you are using any decent DB system it should have some form of profiler for watching the execution of statements. Yes, you could use this to capture your dynamic sql but you are still going to have to recompile the application or assembly to get the changes you require into the application.

<br>

<br>+ Extendability

<br>

<br>How many common SQL statements do you have lying around your application? Hopefully you had some idea that you should probably wrap each statement in a method and reuse that method. Now what if you want to slip in a new business rule or process that affects data integrity?

<br>

<br>For example, say you are writing a system that has &quot;events&quot; that can occur during the lifetime of a product. What if you need to add a new event and the processing associated with it? If the application's data workflow is driven by logically placed stored procedures you could slip it right in without the system even knowing it but at the same time the system would be able to take advantage of it without having to recompile and everything else associated with application change management. Yes, you should test the integration of the new piece of data workflow and in most cases it will be simple to make sure you that you create a side-effect free procedure or clearly identify procedures that have side-effects.

<br>

<br>+ Performance Tuning

<br>

<br>If for some reason you need to horizontally partition a table that has grown too large, how are you going to do that in your application if you arne't using a stored procedure? Well, I suppose you would have to go in and tweak lots of SQL data access code.

<br>

<br>With a stored procedure you could simply replace the data access code without touching the application again.

<br>

<br>I have used mixed stored procedures and embedded SQL statements in my immaturity of application development. Applications that I still support today, and I dread having to fix or tune them because it is painful. I swayed from stored procedures or embedded SQL, at the time it was a time thing, which was the wrong choice. Each module of the appliation showed a new maturity into how data access should be handled and how it shouldn't.

<br>

<br>Whenever someone asks me for examples of best practices, I often refer them to worst practices first. Because, many times people will try to incorporate best practices within their application but some how, simple worst practices slip in over time because they are not aware that it really is a bad idea to actually use a certain technique.

# RE: N-Tier development

left by at 6/4/2004 6:45 AM Gravatar

&quot;Another example is foreign key constraints, I always do lookups to make sure that the keys exist before I do anything to them. By the time I actually perform the DML statement I am 99.999% sure that it is going to succeed.&quot;

<br>

<br>Maybe I'm misunderstanding what you're saying... You're checking for key existance before running executing a CRUD?  Why wouldn't you set the foreign key constraint in the database and just call the CRUD stored proc?  The dbase will enforce integrity and toss and error back in the case something isn't right.

<br>

<br>I apologize if I misread, but it reads like you're explicitly performing a check that most databases will do for you if you configure them properly.

# re: N-Tier development

left by at 6/4/2004 6:58 AM Gravatar

I've been using SPs for all my apps since I first started working with SQL Server. I've even started using them for Access lately. All of the recent buzz across the web about O/R mappers and dynamically generated SQL statements prompted my curiousity and thus my post about what other junky bloggers use. I haven't seen anything on here thus far that warrants me changing from SPs either. I honestly don't know what I'd do without places like this to give me interaction with other programmers (since I'm the only one where I work). I guess all of my questions would go unanswered and my code would never improve. eek!

# re: N-Tier development

left by at 6/4/2004 7:11 AM Gravatar

True, but in most cases I do not want to produce a &quot;database&quot; specific error. I have a set of understood and excepted erros that the data layer can provide. If I receive something outside of this, it is because I missed a data-validation some where or some operation occurred that was fatal to the success of the command.

<br>

<br>So in most cases I am checking for things like, if I am changing a user name, I need to make sure it doesn't already exist to the one they are changing it to so I usually have an EXISTS check that throughs an error.

<br>&lt;code&gt;

<br>IF EXISTS(SELECT * FROM dbo.Users WHERE Name = @name AND UserId &lt;&gt; @userId) BEGIN

<br>  EXEC dbo.Errors_UniqueKeyViolation('Users', @name)

<br>END

<br>&lt;/code&gt;

<br>Yes, I do realize that the database in most cases will provide this for me. Going back to known expected errors I want to control which errors fly about the application. Just another safety net to know that, if this error occurs it will be handled properly. Less unknowns, more stable code.

# CodeSmith... clarification and a retraction

left by at 6/16/2004 6:54 AM Gravatar

CodeSmith... clarification and a retraction

# MyDal...

left by at 7/9/2004 1:41 PM Gravatar

MyDal...

# re: N-Tier development

left by at 7/15/2004 7:26 AM Gravatar

I found a pretty serious flaw in that SQLHelper (Microsoft.ApplicationBlocks.Data) thingy.  I kept getting timeout errors and running out of pool connections until I found this.  If you call the &quot;ExecuteReader&quot; functions by passing a connection string, it creates and opens a connection, then leaves it there.  There is no way to close the connection at that point since you can't get a reference to it.  Next time you call it, it opens a new one instead of re-using the connection.  

<br>

<br>So, the thing to do is create and manage the connection and use the ExecuteReader methods that accept a connection object.

<br>

Title  
Name
Email (never displayed)
Url
Comments   
Please add 4 and 3 and type the answer here: