Download User Manual and Reference

Chapter 6
The Implementation of TPIE
6.1
The Structure of TPIE
TPIE has three main components, the Access Method Interface (AMI), a Block Transfer Engine (BTE)
component, and a Memory Manager (MM) component. Various Block Transfer Engines (BTEs) can be
chosen for handling disk block transfers, perhaps more than one in a single application. The MM component provides low level memory management services such as allocating, deallocating, and accounting
of internal memory. The AMI works on top of the Memory Manager and one or more BTEs to provide
a uniform interface for application programs. Applications that use this interface are portable across
hardware platforms, since they never have to deal with the underlying details of how I/O is performed
on a particular machine. This chapter describes the design decisions, algorithms, and implementation
decisions that were used to build the MM, BTE and AMI components of TPIE. The Reference Section
of this manual contains a description of the AMI and Memory Manager entry points that an application
programmer might normally use. Typically, an application programmer will not request services from
a BTE directly. For this reason, the BTE services are not presented in the Reference Section of this
manual, but are presented here for those readers who wish to understand the implementation details of
TPIE.
The MM manages random access memory on behalf of TPIE. Currently, TPIE is distributed with
an MM designed for a single processor, or multiprocessor system with a single global address space.
This MM is relatively simple; its task is to allocate and manage the physical memory used by the BTE
component.
The AMI is an interface layer between the BTE and user level processes. It implements fundamental
access methods, such as scanning, permutation routing, merging, and distribution. It also provides a
consistent, object-oriented interface to application programs. The key to keeping the AMI simple and
flexible is the fact that its user accessible functions serve more as templates for computation than as
actual problem solving functions. The details of how a computation proceeds within the template is up
to the application programmer, who is responsible for providing the functions that the template applies
to data.
6.2
The Block Transfer Engine (BTE)
The BTE component is intended to bridge the gap between the I/O hardware and the rest of the system.
It is the layer that is ultimately responsible for moving blocks of data from physical disk devices to main
memory and back. It works alongside the traditional buffer cache in a UNIX system. Unlike the buffer
cache, which must support concurrent access to files from multiple address spaces, the BTE is specifically
designed to support high throughput processing of data from secondary memory through a single, user
level address space. In order to efficiently support the merging, distribution, and scanning paradigms,
several BTEs support stream-oriented input and output of data blocks. To further improve performance,
some BTE implementations move data from disk directly into user space rather than using a kernel-level
67