--------------------------------------------------------------------------
T O R Q U E
--------------------------------------------------------------------------

o XML as configuration

o Use the digester to turn the datamodel into an object and
  get rid of the dependency on xerces. It would be nice to
  use the digester with MinML so that the distribution can
  be tiny

o Some analysis on what we actually have would be prudent. Torque
  is somewhat based on the work of Scott Ambler and I would like
  to know where it is the same and where it diverges so we know
  what we're dealing with.

o find all other OSS persistence layer projects and see if
  we can join forces.

o using cursors to allow controlled access to a large set
  of data. typically there is no need to see a massive
  set of records all at once.

  [JSS] Not really database agnostic though. Must be implemented
  conditionally.

  [JVZ] But it has to be made to work transparently. The specifics
  should be in the RDB specific object. The adapter probably.

o using proxies for large objects. if you had a user object
  that had 50 attributes it would more efficient to create
  a proxy object of that user that might have a few attributes,
  enough to identify the object. you might use the proxy
  objects in a list and when it is selected the real user
  object is transparently retrieved.

  [JSS] I consider this custom code and can be implemented in
  the base objects quite easily.

o would it be possible to support native connection methods
  for high efficiency. an easy to access C APIs, would it
  even be useful?

  [JSS] in this case, the JDBC driver can be made native and
  is probably not a worry/optimization of ours because this
  is handled in the JDBC layer.

o application programmers should be able to save, update, and
  delete objects without any knowledge of the database schema.
  if the attribute selection is based on columns in tables than
  the object hierarchy and database schema cannot vary
  independently. it is likely that the database schema may need
  to change but you don't want this to affect application
  programmers who may know little about databases. reasonably
  so, the persistence layer should take care of everything.

  [JSS] YES!

Questions
---------

These are really just questions I'm jotting down as I go through
the source. Not really important that they be answered here, I'll
post the whole slew to the list eventually.

o Why is there a save(DBConnection) in the BaseObject?
  The objects should know nothing about databases, and if knowledge
  of a specific database is required it should probably be in
  some mapping mechanism. The BaseObject should be retrievable
  from any store so the BaseObject shouldn't contain information
  about any specific store type. XML persistence would be nice
  to add with one of myriad tools that exist.

o Same question for the the Persistent interface. Would be nice
  to get rid of the DB specifics.

o Why do we have so many key types. I thought the surrogate key
  idea was better? That the keys for objects should have no business
  meaning under any circumstance. This practice is strongly warned
  against in Ambler's papers. The object key should be a number or
  a string.

o Why does the TableMap make reference to the IDGenerator? I realize
  the id method can be specifed on a table basis. Maybe the datamodel
  object could be used at run time as the source of information about
  the database setup. Than a lot of the objects that are coupled could
  be decoupled because all the meta info would be present in the
  datamodel object.

o Same goes for the DatabaseMap. We definitely have information duplicated
  across objects. Most of this info is already present in the datamodel
  used to produce the OM. Could use the datamodel to decouple which is
  what would be need to have pluggable connection pools.

o Could the information in the adapters be coupled with information
  in the db.props file and combined into some XML descriptor for
  database specifics?

----

Get rid of ${torque.home} in the standard build.xml files and define
all the possible values that can be set and document them so that
people can use the standard build.xml file in a known way. Things
are a little loosy-goosy and fragile right now. What I do could still
potentially break scarab's build because I'm not sure how things
are being used ... but I'm getting there :-)

Scott Eade has found that Monitor threads appear to be living on past
servlet restarts. Need to add a test for that.

----

Problems with Large Schema Files

In some cases, large schema files can cause Torque to fail to generate
the object model.  There appears to be a memory leak which affects
only the generation of the object model.  It does NOT present any
problems at runtime, only during generation and only for especially
large or complex schema files.  (FIXME: measure what "large or
complex" really means -- in units of tables or foreign keys or
something.)

An attempt has been made (unsuccessfully) to allow a schema to be
split across multiple files.  These notes are intended to record the
problems in hopes that solutions can be found later without having to
rediscover the problems.

A schema can be successfully split into multiple files provided that
all foreign keys defined in one file refer only to tables that are
also defined in the same file.  A schema could be sufficiently complex
that even the split up files would be too big for Torque to generate.

One solution would be to find a way for Torque to generate code using
keys to tables defined outside of the schema file.  As of March 8,
2002, this doesn't work.  Object.vm includes some complex logic for
generating methods to get related objects, and even more complex logic
for methods that get related objects by joining multiple tables.  In
order to generate these methods, Object.vm currently goes to the
corresponding Table objects to get their foreign keys.  That is what
constrains the generation.  The ForeignKey object does not contain
sufficient information for Object.vm to generate those methods without
support from the Table objects.  Therefore the Tables must also be
defined and in memory.  Hence the requirement for a single file.

A couple possible areas for investigation.  Perhaps the graph package
in Commons could be employed to capture the foreign key structure in a
more memory efficient way.  Right now Torque uses Texen tasks and
velocity templates to transform the xml schema into object models.
Might be worth investigating DVSL to see if it can do what Texen does
but with less memory (or with less complexity).
