Make Your Database Match Your Program

The installation instructions for most ColdFusion programs include extra steps for creating the tables in the database and more for adding columns for upgrades. It is possible, however, to have your application take these steps automatically.

The DataMgr components can be used to install the needed tables your program needs on any database supported by DataMgr (currently MS Access, MS SQL Server, MySQL and PostGreSQL - but adding a new database is easy). The ability to add needed columns to existing tables is a new feature that has just been added to DataMgr (as is support for PostGreSQL).

In earlier posts, I have covered an introduction to DataMgr and inserts using DataMgr.

To instantiate a Data Manager component, you need to pass in the datasource in the init() method.

<cfset Application.DataMgr = CreateObject("component","com.sebtools.DataMgr_PostGreSQL").init("mydsn")>
You can then pass the DataMgr into your component and use it to create tables and columns as needed. The loadXML() method of the DataMgr will load the required table structure into the DataMgr using an XML format it defines for that purpose and optionally creates tables and columns as needed.
<cfset variables.DataMgr.loadXML(xmlstr,true,true)>
The first argument defines the XML for the table or tables needed by the component. The second argument (which defaults to false) indicates whether or not to create any of the tables that don't exist. The third argument (also defaults to false) indicates whether or not to add needed columns to existing tables.

With this code your component will automatically create any tables and columns that it needs. You can also load an existing table into the DataMgr by using the loadTable() method (which takes the name of the table as its only argument).

If the table is loaded, you can get the XML to create the table using the new getXML() method which returns the XML for a table (or for all tables loaded in the DataMgr component if none is given). You can then put that XML in your component to create the table when the component is loaded later.

This should make it easy to install tables as needed and update them with the new requirements of your program.

You can download the DataMgr components (for free) from http://www.bryantwebconsulting.com/cfcs/.

Give it a try and let me know how it works!

Good luck!

Easier Inserts

When I do a database inserts, a few issues come up repeatedly.

I need to...

  • get the primary key of the inserted record
  • make sure I only insert data for fields that I have
  • only use data of the correct type for my fields
  • check for duplicate entries
None of these issues is particularly difficult, but it is tiresome to code for these issues each time.

So, I built some (free) Data Manager components to handle these issues. I have given an overview of these in a previous blog entry.

I recently updated the components to enhance database inserts. They will now return the primary key of the inserted record and check for duplicate records.

To instantiate a Data Manager component, you need to pass in the datasource in the init() method.
<cfset Application.DataMgr = CreateObject("component","com.sebtools.DataMgr_MSSQL").init("mydsn")>
This example uses the DataMgr for MS SQL with a datasource of "mydsn" ("DataMgr_MSSQL" is for MS SQL Server, it could be replaced as needed - with "DataMgr_Access" for MS Access, for example). The Data Manager components must be in com.sebtools. This can be in a com/sebtools directory under your CustomTags directory or under the root of your web site.

To insert a record, call the insertRecord() method. The method has three arguments:

1: tablename (required): Simply the name of the table into which you want to insert data.
2: data (required): a structure of the data that you want to insert, each key representing a field of the same name.
3: OnExists (optional): what to do if a matching record already exists.

The Data Manager component needs to know the structure of the table in order to act on it. For most databases, it will be able to determine the structure on its own. For MS Access, the Data Manager will try its best to determine the structure, but it must guess on the primary key and is unable recognize an autonumber field on its own (the loadXML() method can be used for this).

When you insert a record, DataMgr will do its best to return the primary key (returning an empty string if it is unable to determine the primary key). In practice, this works well, but it does have some limitations.

Data Manager returns the primary key...
  • reliably if it was included in the insert
  • reliably for SQL Server GUID and Identity fields
  • normally for Acess Autonumber fields, but with some risk of an inaccurate result in very high loads with no locking.
  • normally for MySQL Counter fields, but with some risk of an inaccurate result in very high loads with no locking.
  • reliably if if all of the filterable fields are unique to that record.
In any other event, it may return an empty string if it fails to determine the primary key.

By "filterable fields", I mean those fields that you can test in the WHERE clause of your query.

Since duplicate records can be problematic, the OnExists argument of insertRecord() has the following possible values to handle duplicate records (where all filterable fields match):
  • insert (default): insert the duplicate record (returns the primary key only if passed in or with an incrementing field or a SQL Server GUID).
  • error: throw an error stating that the record already exists.
  • update: update the existing record (returns the primary key of the updated record)
  • skip: don't do anything (returns the primary key of the already existing record)
The Data Manager component currently comes in three flavors (each of which needs DataMgr.cfc to work):
  • DataMgr_Access.cfc for Microsoft Access
  • DataMgr_MSSQL.cfc for Microsoft SQL Server
  • DataMgr_MYSQL.cfc for MySQL
I am working on DataMgr_PostGreSQL.cfc for PostGreSQL as well and I would love to get access to Oracle and Informix databases to make versions for those databases.

Feel free to download the Datamgr components and try them out. I would love to get any feedback on them.

Good luck!

DataMgr Components

The Challenges

Whenever I do a simple insert or update, I do several of the same simple checks.
Beyond that, the error messaged I get from the database often aren't very informative. Specifically, when the data is too long for a string field, the database generally doesn't indicate which field.

I also have a handful of reusable components and I want them to create any tables they need on any database I might use.

The Solution

Let's look at a sample "CritterMgr" component for handling, say, critters. This will be stored in CritterMgr.cfc in the root of our web site.

<cfcomponent>

<cffunction name="init" access="public" returntype="any" output="no"> <cfargument name="DataMgr" type="com.sebtools.DataMgr" required="yes"> <cfset variables.DataMgr = arguments.DataMgr> <cfset variables.DataMgr.loadXML(getDbXml(),true)>

<cfreturn this> </cffunction>

<cffunction name="setCritter" access="public" returntype="void" output="no"> <cfargument name="CritterName" type="string"> <cfargument name="BirthDate" type="date"> <cfargument name="CritterID" type="numeric" required="No">

<cfif StructKeyExists(arguments,"CritterID")> <cfset variables.DataMgr.updateRecord('Critters',arguments)> <cfelse> <cfset variables.DataMgr.insertRecord('Critters',arguments)> </cfif> </cffunction>

<cffunction name="getDbXml" access="private" returntype="string" output="no"> <cfset var tableXML = ""> <cfsavecontent variable="tableXML"> <tables> <table name="Critters"> <field ColumnName="CritterID" CF_DataType="CF_SQL_INTEGER" PrimaryKey="true" Increment="true" /> <field ColumnName="CritterName" CF_DataType="CF_SQL_VARCHAR" Length="60" /> <field ColumnName="BirthDate" CF_DataType="CF_SQL_Date" /> </table> </tables> </cfsavecontent> <cfreturn tableXML> </cffunction> </cfcomponent>

We will initialize CritterMgr with the init() method. Since that takes the DataMgr as an argument, we better initialize that first (Application scope is personal preference only):

<cfset Application.DataMgr = CreateObject("component","com.sebtools.DataMgr_MSSQL").init("mydsn")>

CritterMgr expects a type of "com.sebtools.DataMgr", but we are instantiating com.sebtools.DataMgr_MSSQL. This works because DataMgr_MSSQL extends DataMgr. Note that DataMgr components must be in com.sebtools. This can be in a com/sebtools under your CustomTags directory or under the root of your web site.

Now we can call the CritterMgr:

<cfset Application.Critter = CreateObject("component","CritterMgr").init(Application.DataMgr)>

CritterMgr's init() method takes the DataMgr and makes it available to the rest of the component in variables scope.

<cfset variables.DataMgr = arguments.DataMgr>

DataMgr must know the structure of any table that it works with. If the tables already exist, you have the option to simply pass the table name to the loadTable() method (note that DataMgr_Access does not yet support this).

Our CritterMgr, however, needs to define its own tables, so we use the loadXML() method of DataMgr. CritterMgr passes the XML to DataMgr's loadXML() and indicates true for table creation. In this example, the XML is retrieved from a private getDbXml() method.

<cfset variables.DataMgr.loadXML(getDbXml(),true)>

This should be a good example of the XML format. The XML Schema is also available. Note that the XML is case-sensitive.

In this example, we are creating tables in MS SQL Server (by using DataMgr_MSSQL). If we wanted to install the tables in a MySQL database we would have just called DataMgr using DataMgr_MySQL. DataMgr components exit for Access, MS SQL and MySQL (so far).

Next we need to add and update a records. Let's assume that we have a form with a BirthDate field and a CritterName field. The form will also contain a "CritterID" field if it is for an update. That being the case, here would be our code for an insert or update on the action page.

<cfset Application.Critter.setCritter(argumentCollection=form)>

Or

<cfinvoke component="#Application.Critter#" method="setCritter" argumentcollection="#form#"></cfinvoke>

Since the form scope is a structure, we can pass it into the setCritter method using argumentcollection. In this case, the form names match the argument names. Otherwise, we would have to specify each field instead of using argumentcollection.

The setCritter() method then calls insertRecord() or updateRecord() appropriately. It passes in the name of the table on which to perform the action and the structure of data to insert/update.

<cfif StructKeyExists(arguments,"CritterID")>
    <cfset variables.DataMgr.updateRecord('Critters',arguments)>
<cfelse>
    <cfset variables.DataMgr.insertRecord('Critters',arguments)>
</cfif>
For inserts and updates, DataMgr needs a structure indicating the data to insert or update with a key for each field with the value that you wish to insert/update. Since the arguments scope is a structure, we can pass it directly into our insert or update method. If the argument names didn't match the field names in the database, we would just need to create the appropriate structure in the method itself.

DataMgr will test each value for validity with the data-type of the field. It will throw ane error for any character field receiving more data than it can hold. Note that updates must include the primary key field or the DataMgr will throw an error.

Any fields missing from the insert/update method will not be included in the resulting query. So, if the BirthDate field isn't included in an update then it would keep the value it had before.

Conclusion

Now, inserts and updates are handled easily and with extra error-handling. Our CritterMgr can easily be installed on any database which DataMgr supports.

For more information, check out my CFCs page, which has documentation on DataMgr and downloadable files.

Leave me a note and let me know what you think!

BlogCFC was created by Raymond Camden. This blog is running version 5.8.001.