Re: Data warehousing and XML

len bullard (cbullard@hiwaay.net)
Mon, 01 Dec 1997 19:14:26 -0600

Paul Prescod wrote:
>
> Simon St.Laurent wrote:
> >
> > This may be writing it off too quickly; I think the great advantage of XML for
> > a data warehouse would be its ability to ease the inclusion of non-relational
> > data.
>
> I don't claim to be an expert, but my understanding was that the goal of
> a data warehouse was to make a repository of information that could be
> plumbed through essentially relational queries -- demographic
> information, correlations between dates and so forth. I don't see the
> benefit of including document data in this sort of repository. As
> someone else mentioned, XML may well be a good transfer format for the
> information to move it FROM the regular databases into the data
> warehouse.

Of course XML can be used to create non-ambiguous
transfer formats (data schlepping). But Paul,
a lot of the information that needs to be mined
is not in relational formats. Depending on the query
language and implementation, there is no reason one
cannot build an industrial strength data repository
over generalized markup. Some IETM applications
(eg, 87269, MID, etc.) are designed to do that.
Even with IADS some years ago, we had some primitive
capabilities for this although immature then. Probably
much improved now. In those designs it was always
assumed that the client language (eg, MID) was
essentially a navigation system over a set of
notations whose processors are known. It is also
assumed that the client language included a query
language or could call one. So, data warehouse may
be in need of further clarification. Applications
I work with have to have both document frameworks
and relational systems as well as all of the
ad hoc-inTransit data used to interface the
live data (sensor-derived) to the database that
is collecting and warehousing.

However, let me ask a technical
question that you can probably answer with a deeper
technical perspective than mine? How well can one query
data (or convert it for that matter) for which one
has no rigorous schema (of some kind)? (Note,
I consider a self-identifying type (eg, magic number)
to be a pre-validated file of the notation.)

len bullard