ࡱ> ghfܥhc eMdaF....... :=1""""""XZZZE_nX==."XY """"=D.."DDD""."."X@΁CB bh...."XDDData Ware-house Case Study Jonathan Lewis UKOUG Conference 1997 1.0 Introduction: This article uses a project that the author has been involved with as a background for outlining a number of technical and managerial issues that need to be addressed during the development of a Data Warehouse. Since the term Data Warehouse covers a multitude of sins I should explain that the context for this article is a large collection of historical data used to identify sales and wastage patterns and evaluate the success of marketing strategies. As such this Data Warehouse is little more than a single large table of around 1 billion rows that is essentially a read-only table subject to a batch insert each night. 2.0 The Database: The database is one module of a system designed for a large retail organisation. The raw data arrives daily from an EPOS (electronic point of sale) system. There are some 500 stores of varying sizes and each store supplies details of its daily total sales and wastage across a range of about 20,000 active products. Some data, around 2 or 3 per cent of the total volume, arrive up to 28 days late - although most late arrivals appear within 48 hours. Some stores sell only a restricted range of products, and not every store sells every product every day, so the nominal 10M combinations of store and product is reduced to a typical volume of around 2M rows per day. Daily data is stored for a maximum of 495 days, giving a raw data size just under 1,000,000,000 rows; weekly and period-based summaries are stored for 2 years. There are three dimensions to the data that the user could select and summarise: Time: e.g. day, week, period (month), year to date Store: there are several types of classification, e.g. TV region, Sales region, County Product: again there are several types of classification; there is also an important product hierarchy that runs to 6 layers. One of the most challenging features of the database is that this product hierarchy is the most important access path to the data, but it can change over time and the users want historical data to be summarised using the current hierarchy. sum sales and wastage of sun-cream by week for the last 8 weeks for each store in the Carlton TV region showing wastage as a percentage of sales sum the sales of mens suits daily in Yorkshire over the last two weeks separated into 3-piece and 2-piece match against the corresponding two weeks last year showing the percentage uplift from last year to this year Fig 1: Sample Queries 3.0 The Application Given the purpose of this database there is no application as such, there are only data query tools - currently Oracles Discoverer Vn. 3, and Powerbuilder Vn. 5. Discoverer is an almost pure end-user query tool that writes its own SQL statements based on point and click, drag and drop manipulation by the user. To protect the database from the worst possible accidents there is an End User Layer that can be pre-defined so that the user is restricted in the choice of tables they can access and the way in which the tables may be joined. The benefit of Discoverer is its great flexibility, in particular its ability to drill down from the results of one query to produce underlying detail. A useful technical feature of Discoverer is that it can be configured to re-direct queries to pre-prepared summary tables transparently delivering better performance to end-users. There are currently three major deficiencies in Discover: first the presentation layer cannot derive columns from retrieved data, all such work has to be done by the database; secondly it cannot include (useful) hints in the generated code; finally it cannot cope with UNIONs. Powerbuilder is used as a pre-programmed access tool. Any SQL that arrives at the database from Powerbuilder has been generated by code written by in-house programmers. Users are restricted to calling up pre-designed query screens, supplying parameters to pre-defined queries, and drilling down through specific paths. The benefit of Powerbuilder is that the code arriving at the database is strongly controlled; the drawback is that new classes of query may require significant programmer time. 4.0 The Environment: The database runs under Oracle 7.3.3.3 on a Pyramid RM1000 Mesh running Reliant UNIX. This is a parallel server system and 10 instances are assigned to the data warehouse. The discs are all 4GB and most are configured as RAID-5 although each nodes system disc is a simple, non-striped, non-mirrored disc. Disc access is through a Logical Volume Manager which superimposes raw devices on top of the RAID-5. Redo is mirrored by Oracle onto simple raw disk; TEMP is on simple non-mirrored disc. Each node is responsible for a chain of 6 discs, but all the discs are dual ported and each node has a buddy node that can take over its SCSI chain if the node fails. All nodes can see all discs on the system. Access to remote discs (and messages between nodes) are made through the Mesh, a proprietary hardware 2-dimensional network operating at very high speed. To maintain cache coherency, i.e. to make sure that two different instances do not attempt conflicting updates to the same data block, the Oracle instances communicate with each other using a Distributed Lock Manager (DLM) supplied by Pyramid but defined by Oracle. 5.0 The Business Needs: Although a lot of work had been done on the business requirement, much of it revolved around give us what we already get from other systems, and there was still plenty of room for uncertainty. Going into the final design phase, the most firmly indicated needs were: Total flexibility Queries would almost invariably be based on the current product hierarchy A large fraction of queries would be based on week or period summaries. There were some special cases which would have to be addressed separately A query could span a completely arbitrary time range across the 495 days, or 104 weeks, or 26 periods. The most significant design challenge in this short list is the second one - Since most queries were to be satisfied by summarising according to current hierarchies, there seemed to be little point in trying to have pre-aggregated values (other than time-based ones) stored in the database as the time required to rebuild an aggregate when the product hierarchy changed would be in excess of the overnight window. 6.0 The Design Strategy: As a project manager, there are three important guidelines: Keep it simple. Make a few reasonable guesses. Wait (and budget) for evolutionary pressure. In our case the reasonable guesses led us to collect all the data without pre-aggregating any of it (other than into weekly and period summaries). We didnt denormalise any of the dimension data into the fact data. The only concession we made to data-warehousing was to flatten any dimensional tables as much as we could. Since we expected most queries to collect a very large number of rows, we then tried to produce a simple physical layout that would gear the system to a very high I/O throughput, maximising the number of devices that would be accessed by any one query and minimising the number of Oracle blocks that needed to be visited for the query. Discoverer has features for collecting information about the queries it generates, so this could give some indications about the types of aggregation strategies that would be needed as the system became more heavily used. We would build pre-aggregations only by common demand. 7.0 The Technical Solution: 7.1 Operating System Level: The technical solution was to set the system up with RAID-5 with a stripe size (i.e. space per device) set between 16K and 128K, then collect 4 RAID-5 devices into a raid-set. A single data file would then be created as a logical volume that was defined to circulate around the 4 parts of the raid set writing 2M on each RAID-5 before moving on the next. In this way a single file was spread evenly over 20 different disks. Since each daily table was about 80M, this gave us the benefit of spreading each table very finely without an unpleasant administrative overhead: most of the complexity was handled by the LVM. Since we had set up 3 raid-sets (i.e. 60 discs in total) we spread the I/O further by creating 4 tablespaces per period:: daily data daily indexes week/period data week/period indexes and separating data tablespaces as much as possible from the corresponding index tablespaces. Ultimately a typical nested loop in parallel into the data would spread its I/O fairly evenly over 40 different disks. The success of this strategy was such that at peak rates we have seen a query identify, acquire and summarise 8.5M rows and produce a one page report in 3.5 minutes. A parallel tablescan to summarise 2,000,000 rows and 80M of data takes about 10.5 seconds. 7.2 Oracle Level: An important feature to consider is the clustering of the data. In recognition of the high probability that the typical data access would be product based we pre-sorted the data by product code before loading it into the database. We ensured that the weekly and period summaries were clustered in the same way by creating them in a two-stage process - a rapid, parallel, sum into a holding table followed by an indexed serial copy to create the final sorted table. The effect of this is to make the product based index a very good index, and highly likely to be chosen as an access path by the cost-based optimiser (CBO): the clustering factor of the index is so good that a query for all of product X from a single daily table is almost certain to be met from a single table I/O. A side-effect of this sorting is that the obvious store-based index is appalling - in fact we are currently planning to drop ours and reclaim about 30% of the total database space. Whilst talking about space, it is worth noting that some of our recent work on bitmap indexes suggests that we could safely convert the product-based indexes to bitmap indexes with no damaging performance side effects (possibly with a benefit) and reclaim a further 25% of the database space. Apart from data arrangement, the other main concern for physical implementation of such a system is how to handle very large objects, and how to deal with the apparent need for massive backups. The solution in both cases comes from Partition Views. 8.0 Partition views: Create a set of 26 identical tables. Add a constraint to each table to restrict the data in that table to a specific time period, e.g.: check (year = 1997 and period = 1) Create a VIEW that uses the UNION ALL set operation to concatenate the set of tables. The result is a PARTITION VIEW. Create or replace view all_periods as select * from period_1996_09 union all select * from period _1996_10 union all select * from period_1998_08; This is a new feature of Oracle 7.3 that allows a single large object to be split down into a number of small objects that can be handled efficiently and effectively by the DBA. There are two important features to the partition view: first a query that includes a literal test on the constraining columns allows Oracle to avoid searching unnecessary tables. In the query below Oracle will only examine ONE table. The other twenty-five will be eliminated in the parsing stage: select sum(sales) from all_periods where product_code = 123 and year = 1998 and period = 2; The second important feature about partition views is that Oracle will allow join conditions to be folded into the view. In the next query Oracle will effectively join the Product table to each relevant data table in turn, then put the three sets of results together. By comparison, if the view were NOT a partition view, then Oracle would turn the whole view into an internal temporary table and spend a huge amount of time joining the resulting millions of rows - without an index - to the product table. Select p.product_name, sum(v.sales) from all_periods v, products p where p.product_class = Spanner and v.product_code = p.product_code and v.year = 1998 and v.period in (1,2,3) group by product_name; The benefits that we gain from the special features of Partition Views are enormous. In the first place we no longer have to worry about the awful maintenance costs of very large objects - no more need to rebuild an index on a billion rows, for instance. Secondly the indexes we have to create do not need to include any information about the partitioning columns. In the past a table with all 26 periods in it would almost certainly contain the two columns (year, period) in every index. The space saving from this change alone is significant. Third loading up large amounts of new data to append to existing very large tables is no longer a problem. A common question in the past was: What is the best way to insert another 2 million rows into a table holding 20 million when I have 4 indexes on the table? With partition views the question does not arise. To add the next days data you simply use SQL*Load to direct load a new table, create (unrecoverably) the appropriate indexes, then replace the current view with a new view to include the latest table (and drop off the oldest table - no more need to ask What is the best way to delete 2 million ....? ) Finally, the problems of backup and recovery suddenly become much more manageable. 9.0 Backup and Recovery: One of the business constraints on the system was that some data would arrive up to 28 days late. But look at this statement from the opposite direction - any data more than 28 days old is not supposed to change. If such data really does not change then we can take advantage of Oracles READ ONLY tablespaces. First we create the obvious high-activity tablespaces such as SYSTEM, RBS, REF_DATA (data and indexes) which would be critical to a rapid recovery. We also need to have a few temporary tablespaces such as TEMP (a proper temporary tablespace for internal use), SALES_SCRATCH, and WASTAGE_SCRATCH; if we were courageous we might decide that we could survive without backing these up since a possible recovery strategy would be to perform an OFFLINE DROP if needed, followed by a create tablespace. Finally we look at the real data, of which only the last 28 days needs to be read/write. An obvious strategy for handling this data in convenient chunks is to break it down into periods (A period is usually 4, but sometimes 5, weeks). As I mentioned section 7.1, we have 4 tablespaces per period (daily data, daily indexes, summary data, summary indexes). Of these only the two most recent sets of tablespaces are read/write; all the others are READ ONLY. At the end of each period another set of tablespaces is made read only, backed up, and added to our library of historical tapes. The daily (cold) backup is minimal - from a 200 GB database we back up less than 16Gb. For the purposes of a quick turn-around time, and fast recovery, this 16Gb is first backed up to disc whilst the database is down, then copied to tape after the database is restarted. The typical cycle time is less than 45 minutes. An interesting benefit of this approach is that we can build a Test database simply by recovering the 16Gb of backup, adding a couple of TEMP and SCRATCH tablespaces, then picking a handful of the history tablespaces from the library. 10.0 What Happened Next?: I highlighted in section 6 the need to start with something simple and wait for evolutionary pressure - so how did our system have to evolve after it went live? All reference/hierarchy tables were denormalised, so almost all queries needed a maximum of 3 tables (product flattened, store flattened, data). Because of the way we packed and indexed the data, the CBO usually took a path which scanned the Product table in parallel using the results to perform a parallel, index-based, nested loop join (see fig.2 below) into the data table, finally hashing the intermediate results with the store table to eliminate unwanted stores.  Fig.2: Parallel scan with indexed NLJ Initially this gave excellent performance. Fortunately the users were very reasonable about recognising the sheer volume of data that some of their queries required and did not demand sub-second response time for DSS queries. However as usage and sophistication grew the nature of the queries became more complex and covered wider ranges of data. Eventually we had to pre-aggregate up the product hierarchy, even though this meant recalculating vast volumes of data regularly. We chose a level half way up the hierarchy to collapse 2M rows to about 70,000 rows - higher would have been too dense for many queries and required too much further data to be accessed on drill down, lower would have taken too long to summarise and not given enough of a performance benefit at query time. The chart below gives an indication of the how the data for a single day would compress if it were to be summarised up the hierarchy (keeping the store codes in the summation) - my rule of thumb is that collapsing by a factor less than 25 is not likely to be particularly useful:  Fig. 3: Rows per product level An important consideration in devising this compromise was to ensure that we knew the functions of all the modules in the overnight batch so that we could optimise the scheduling of the necessary tasks and avoid duplicating work. We also had to adopt a two-phase strategy that applied rapid changes (by inserts only) to the summaries during the week, coupled with a rebuild at the week-end to reduce the data inefficiencies that had built up over the week as the row counts had increased and data had become scattered. Apart from this genuinely evolutionary change, we are now reviewing some other changes that require us to do a one-off rebuild of large amounts of data and a rewrite of the over-night batch to improve CPU usage. This requirement is partly due to the expansion of the system, and partly due to problems that arose in the late stages in the original build. 11.0 Problems and Errors: Inevitably there are going to be problems during the lifetime of the warehouse, and you will make all sorts of mistakes as you design or evolve your system - so by now you might be wondering what problems I am prepared to admit to. Initially the main problem was the quality of the software supplied by Oracle. The design of the system was heavily dependent on partition views and the parallel query option. Critically: partitions HAD to be eliminated correctly and Parallel Query slaves HAD to get a valid rewrite of incoming queries. The number of serious bugs in the relevant code (especially the Parallel Query code) was significant and we spent a lot of time devising workarounds or building small test cases to pass back to the developers. The speed with which we got fixes was not improved by the fact that we were also running Parallel Server, which is a special case in its own right. We also found it necessary to produce a set of regression tests to find out the side-effects of the fixes we received as some of our workarounds suddenly became liabilities as the Oracle server code changed. The second significant problem was a result, or side-effect, of mixing partition views with the parallel server. The parse time for a 495 table view (needed to allow arbitrary yet easy access to any range of dates) was about 20 seconds when tested serially on a non-parallel system and about 40 seconds when running under the parallel query option. On our parallel server system it was liable to rocket up to 4 minutes due to a minor detail of breakable parse locks that did not come apparent until too late. Because of this parse cost we will be rebuilding our single day tables into tables holding 7 days just before making them read-only. On the design side, we have to review a feature that leaves data in a simple form and presents it to the users through views that use DECODE()s. Recent developments in the user behaviour mean that the cost of some important queries is actually heavily impacted by the amount of decoding that takes place. We are considering a partial data rebuild that trades a few hundred megabytes of null columns to improve response by an estimated 50%. There is room to improve performance by adjusting the RAID-5 stripe size (relatively easy to do) and also by changing the Oracle blocks size (a much more onerous task). It is unfortunate that in most cases you are unlikely to have the time to perform a number of full-scale tests of the conflicting effects of changing these two fairly critical factors in the physical set-up. It is important to note, in particular, that big Oracle blocks are not automatically the best choice for large databases. Most of our data access is by index, so if a single datablock holds MUCH more than the data needed for one index value, then the block size is arguably too big. At present we have 16K blocks when a 4K block would hold all the data for a product in almost all cases: arguably we are wasting 12K of db_block_buffer space per access - potentially we could improve performance quite considerably by rebuilding the database. Finally, a restriction on Partition Views: there is one way in which partition views do not match up to the efficiency of a single large table. Consider the query: compare the sales leading up to Xmas this year with those leading up to Xmas last year. Or in its SQL form select product_code, sale_date, sum(sales) from all_daily_sales where sale_date between {date_1} and {date_2} or sale_date between {date_3} and {date_4} group by product_code, sale_date; This type of query would be of great interest to any retail organisation, and naturally our table all_daily_sales is a partition view partitioned by sale_date to make it efficient. But partition elimination does not work on this type of query - nor, apparently, is it in the 7.3 spec for partitioned views so it is not a bug, nor, so I have been told, is it going to work for partitioned views in Oracle 8.0. Unfortunately, we do not yet have a work-around to this problem as Discoverer cannot handle UNIONs (which was one possible work-around) and the cost of a USE_CONCAT hint on the view definition (another possible work-around) is precluded by the problem of parse costs on a parallel server. 12.0 Summary: In summary then, what lessons are there in this development that could be of general use: There are two lists of answers, one for technicians and one for managers. At first glance the lists appear to overlap, however the arguments behind similar bullet points are quite different. 12.1 Technical Summary Dont try to get it right first time It will be wrong, too complicated, and you wont have time to remove the redundant bits later on. Test to Scale Right size tables, right number of table in partition views, right number of nodes in a parallel server. Think about physical data distribution It can make a very big difference to at least one data access path - choose the right one. Know your front-end Its limitations may have a big impact on what you can allow the database engine to do. Document Workarounds Some of them will be undesirable but the least worst option - tidy them up as soon as possible. Maintain a Regression Test Suite for Oracle upgrades There will be changes to the Oracle code that will make some of your workarounds stop working. Modularise batches as much as possible As you evolve summary code you will want to maximise your options for parallel batches and avoid repeating or waiting for work to complete. Period based read-only/backup measures Look for efficient options for minimising the volume of backups. Prove the backup/recovery cycle. A good recovery strategy will also give you an easy path to a proper full-scale, test bed. 12.2 Managerial Summary: Dont try to deliver too much up front It will be wrong, and it will fill the overnight window so that there will be no time for processing future requirements. Budget for evolution Expect users to want lots more - start cheap and simple and budget more for the future cost of moving to meet discovered requirements. Understand the impact of the end-user tools You must be able to present the cost benefit equation of different types of front-ends in a way that the user can comprehend. The trade will be ease vs. performance vs. development time. Dont let the users forget the volume of data. Its too easy for a user to demand unrealistic performance if you do not remind them how much base data they are looking at and what benefits they are getting. Jonathan Lewis A Data Warehouse Case Study UKOUG Conference 1997 .A ::b67'' -& *  &&TNPPb=Py & TNPP &&TNPP    a :-- & & &--  P@Times New Romank~wWw -.  2 u@2nd June```0K``U & & P@Times New Romank~wWw Z -.  >2 o@3rd June`@`0K``U & &@Times New Romank~wWw -.  >2 Productk@```U5 & P@Times New Romank~wWw [ -.  >2 {@1st June`K50K``U & & P@Times New Romank~wWw -.  t2 i@4th June`5`0K``U & &-4`44a@ &--"Systemw{f`  -@-  $  $  $  $##  $;;CC  $[[cc  ${{  $  $ --'--@-  $  $  $  $  $..66  $NNVV  $nnvv  $  $ --'--B( ww-   &@Times New Romank~wWw - .  t 2 P000k```@Times New Romank~wWw - .  t 2 P001k```@Times New Romank~wWw - .  t 2 P002k``` -- -@-qq--'-  $qK@q --@ -qq--'-  $qK`q --@,-qq&--'-  $qKq &@Times New Romank~wWw - .  q& 2 :ScankUU`@Times New Romank~wWw7 - .  `k2 * Indexed@``U`U`@Times New Romank~wWw -.  `k2 *Indexed@``U`U`@Times New Romank~wWw7 -.  `k2 *Indexed@``U`U`--B( ww- 4!-B( ww- 4q-B( - 4!`-B( ww- 0  0 & ---"3-3--'-  $ `_ --3`-3 `--'-  $Oh0Y --@3-3--'-  $n`S --?3~-37~--'-  $/I0e & & --@-  $  $  $  $##  $;;CC  $[[cc  ${{  $  $ --'--B( ww-  --@-  $||  $||  $||  $||##  $;|;|CC  $[|[|cc  ${|{|  $||  $|| --'--@-  $$$  $$$  $$$  $$$  $$55=$=  $$UU]$]  $$uu}$}  $$$  $$$ --'--@-  $  $  $  $  $((00  $HHPP  $hhpp  $  $ --'--@-  $  $  $  $  $55==  $UU]]  $uu}}  $  $ --'--@-  $  $  $  $  $..66  $NNVV  $nnvv  $  $ --'--@-  $  $  $  $  $((00  $HHPP  $hhpp  $  $ --'--@-  $  $  $  $  $((00  $HHPP  $hhpp  $  $ --'--B( ww-     & --H3I-3@I--'-  $6Qy e -- @3-3@--'-  $p_ -- @3-1@@--'-  $@f@Y --@3-3@S--'-  $*Qu`S & & --@-  $D<<D  $D<<D  $D<<D  $D<<#D#  $D;<;<CDC  $D[<[<cDc  $D{<{<D  $D<<D  $D<<D --'--B( - @ --@-  $  $  $  $##  $;;CC  $[[cc  ${{  $  $ --'--@-  $  $  $  $  $55==  $UU]]  $uu}}  $  $ --'--@-  $||  $||  $||  $||  $(|(|00  $H|H|PP  $h|h|pp  $||  $|| --'--@-  $d\\d  $d\\d  $d\\d  $d\\d  $d5\5\=d=  $dU\U\]d]  $du\u\}d}  $d\\d  $d\\d --'--@-  $$$  $$$  $$$  $$$  $$..6$6  $$NNV$V  $$nnv$v  $$$  $$$ --'--@-  $D<<D  $D<<D  $D<<D  $D<<D  $D(<(<0D0  $DH<H<PDP  $Dh<h<pDp  $D<<D  $D<<D --'--@-  $||  $||  $||  $||  $(|(|00  $H|H|PP  $h|h|pp  $||  $|| --'--B( - a !` A & --3-3--'-  $ e -- @3-3--'-  $;_ -- @3-1--'-  $Y --@3-3--'-  $S & & & &&TNPP &--p: G4=S("  & &&TNPPb=Pyc & TNPP &&TNPP    !-- & & & &--  |8 0 Times New Romank~wWw  -.   2 Deptcp> --"Systemwft  -{-{--'--{-{--'--  VN Times New Romank~wWw 8 -.  2 Section}cc?>pp ------'-----'--  \sk Times New Romank~wWw  -.  2 Class?cWW ---x[-[p--'--x[-[p--'--  p Times New Romank~wWw < -.  p2 aGroupJppp ------'-----'--  < Times New Romank~wWw - .   2 SetG}c> ---X>;-;P6--'--X ;-;P--'--  P Times New Romank~wWw @ - .  P2 AProduct}Jpppd> & Times New Romank~wWw  - .  2 7,000p8ppp Times New Romank~wWw A - .  J}2 8,000p8ppp Times New Romank~wWw  - .  J}2 P70,000pp8ppp Times New Romank~wWw B -.  J}2 a200,000ppp8ppp Times New Romank~wWw  -.  J}2 500,000ppp8ppp Times New Romank~wWw C -.  J}2 A 2,000,000p8ppp8ppp & & &&TNPP &---AB`rM W x      % 7OUm %&+,445$6W7777?;E;??;B>??c@d@;B=BcBdBGCHCED88888888888K88888888888888K888cC c888<8c& ' ( ) 1%8c& ' ( ) 1% & ' ( ) 1%-/#EDzE{EFFFFGGHH#J=J%K&KMMNNQQRRRTSTuVvVWWNWxWyWWWWW888 888888K88 8888 888888 88888888888& ' ( ) 1%-/<A& ' ( ) 1%A& ' ( ) 1%#WWX X6XYXZX[XYYYY[([[[D\E\F\]\\\\\]]^]]]]]L^M^b^^^^Y_Z__ ``5`888888888888K8888888888888888888888888888& ' ( ) 1%-/)5`v`w```` a4aaaaKbLbxb4c5cdcddddJdKdLdMd888888888888888888K @ Normal ]a c.@. Heading 1 < U]ck*@* Heading 2 <UV]"A@"Default Paragraph Font @ Header 9r  @ Footer 9r  "@ CaptionxxUMaMd#"###` 8d +6EAKXMa|_  EHX9 +2EDW5`MdYZ[\]^_`aMaHdjUUaJonathan Lewis0C:\USERS\jonathan\website\todo\UKOUG 97 Full.docJonathan Lewis0C:\USERS\jonathan\website\todo\UKOUG 97 Full.docJonathan Lewis0C:\USERS\jonathan\website\todo\UKOUG 97 Full.docJonathan Lewis C:\USERS\JONATHAN\WEBSITE\dw.docJonathan Lewis C:\USERS\JONATHAN\WEBSITE\dw.docJonathan Lewis C:\USERS\JONATHAN\WEBSITE\dw.docJonathan Lewis C:\USERS\JONATHAN\WEBSITE\dw.docJonathan Lewis C:\USERS\JONATHAN\WEBSITE\dw.docJonathan Lewis C:\USERS\JONATHAN\WEBSITE\dw.docJonathan Lewis"C:\USERS\JONATHAN\WEBSITE\temp.doc@HP OfficeJet LX Printer #2LPT1:OFFICJETHP OfficeJet LX Printer #2HP OfficeJet LX Printer, ,,A^| HP OfficeJet LX Printer, ,,A^| 1Times New Roman Symbol &Arial"h1&1&AP(hR*!= Introduction:Jonathan LewisJonathan Lewis  !"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdejrRoot Entry F@QyY0=@΁Ci+00WordDocumentF4646516mF<@,F:\W\Desktop*.*CompObjjSummaryInformation(  FMicrosoft Word Document MSWordDocWord.Document.69qOh+'0 , T ` l xIntroduction:WEcJonathan Lewis Normal.dotJonathan Lewis2Microsoft Word for Windows 9DocumentSummaryInformation8 ՜.+,0HPpx  JL Computer Consultancy*( Introduction:5@@N(Ivϼ@C@CP՜.+,0HPpx  JL Computer Consultancy*( Introduction: