CBLibrary: The Loader module

Introduction

As has often been observed, the Toolbox makes it very easy to save or export data (through the SaveAs object), but not to load or import data. The Loader module of CBLibrary is my own solution to plugging this gap in the C programmer's armoury.

A conventional Wimp program would have single DataLoad, DataSave and DataOpen handling routines which are used for many different filetypes and contexts. With a complex program these routines quickly become non-reusable spaghetti code, mainly because all the esoteric application-specific code is mixed up with generic dialogue-handling operation.

Acorn's Event library goes some way toward improving this situation, allowing different message handlers to be registered 'for' individual user interface elements (so long as this association is enforced by the programmer). However, this solution has the disadvantage that any generic dialogue-handling code is now fragmented and replicated across different parts of the program.

The general idea of the Loader module is that it provides a much higher-level, structured approach to data-transfer. A client application can simply register and deregister an interest (a 'listener') for particular drop locations/file types, and a handler to be called when data is received. Varying degrees of automatic operation are possible, with the client either allowing Loader to deal with the entire operation itself, or specifying a function to be used to load data from file.

Loader has two significant dependencies within CBLibrary; the Loader2 module (which implements the receiver's half of the data transfer protocol) and the Scheduler module (which is used to abandon stalled data transfers). The Loader2 module is more flexible than Loader; it can be used where there is a preamble to the data transfer protocol (e.g. clipboard paste or drag-and-drop). However, Loader2 is is not documented here.

Required libraries

Apart from reliance on other CBLibrary modules, the Loader module requires the Flex library to manage dynamic allocation of memory for newly loaded data, and the Event library to allow it to register handlers for the messages of the data transfer protocol on behalf of the client. The Toolbox library is also required, to allow incoming DataSave messages to be filtered according to drop areas specified in terms of gadgets within a Window object.

Multiple listeners

Listeners are uniquely identified by their dropzone (file type, window id, gadget id). This information is used to find the correct listener to remove, when the client calls loader_deregister_listener. For this reason, attempting to register more than one listener for a given dropzone will cause a failed assertion (or it will be ignored, if CBLibrary was compiled with NDEBUG defined).

Despite the similar syntax to registering an event handler, listeners work at a higher level. The client cannot register many listeners that choose to claim or pass on load requests. This filtering is all done internally by the Loader module.

The only case where many listeners may be eligible for a load request is for filetype claimants (e.g. listeners registered with the LISTENER_CLAIM flag set). When a DataOpen message is received, the list of currently registered listeners is scanned for one that claims that filetype.

Potentially there may be more than one registered claimant, therefore priority becomes important. The scan priority for listeners is the same as that for event handlers. The most recently registered listener is highest priority, and the least recently registered is lowest.

This could be used to implement the following system: Initially file double-clicks are treated as equivalent to drags to the iconbar icon (opening a window). Subsequent double-clicks are intercepted by a listener registered for the new window.

Errors and the message file

Although it is generally good practice for library code to propagate errors up to the client application, this is often not practical in the case of the Loader module. Because it operates largely independently from the client application, many possible error/report situations must be dealt with internally.

In particular, it will be Loader that communicates with the human user when they make an inappropriate attempt to load a file. A flags word may be passed to loader_initialise, to provide the client application with some degree of control over this behaviour,

It is expected that certain messages should be available from the client application's messages file. These are as follows (suggested English messages are supplied with the example application):

Limitations

In case of direct data transfer from another application, a LoaderFinishedHandler function will usually be passed a pointer to the leaf name that was received with the original DataSave message. However, in rare cases this will not be possible.

The reason is that there may be an indeterminate delay between a data-loading task sending a DataSaveAck message and receiving a DataLoad message in reply (e.g. from a web browser that is downloading a file from the internet). Alternatively, the data-saving task may never reply to the DataSaveAck message at all (e.g. if it is probing for a file path and does not actually have any data to send.)

To prevent memory leaking whenever no reply is received to one of our DataSaveAck messages, stalled load operations are abandoned after 30 seconds. Any DataLoad message that is subsequently received will not be associated with the earlier DataSave message, and the LoaderFinishedHandler will get "<untitled>" instead of a leaf name.

Quitting applications

Because the Loader module is compiled into your application and all its internal data structures are stored in application space, you do not need to deregister listeners before your program quits.

Although internally Loader uses the Event library extensively, it does not need to deregister its message handlers before quitting (for similar reasons).

Initialisation

loader_initialise

const _kernel_oserror *loader_initialise(unsigned int flags);

You must initialise the Loader module by calling this at the beginning of your program, before any Listeners are registered. An application should not attempt to call loader_initialise() more than once, or else the memory previously allocated for linked lists will be wasted.

When invoked, this function initialises the listener list and message handlers list (used internally), and sets the global flags word.

It also sets up permanent Wimp message handlers to listen for DataLoad, DataSave and DataOpen. These are used to intercept requests for the client task to load data, if the file type, window and icon handles cited in the message correspond to a listener registered by the client program.

The Event library's current poll mask is forced to allow UserMessage, UserMessageRecorded and UserMessageAcknowledge. Blocking any of these would interfere with the operation of the Loader module.

It is vital that your program calls loader_initialise() before loader2_initialise() or entity_initialise(). Otherwise the Loader module will intercept DataSave and DataLoad messages intended for the Loader2 or Entity modules (because of the order in which the message handlers were registered).

Flags word

Only bits 0-3 are currently defined. The flags word should be constructed by ORing together the C pre-processor symbols defined by the "Loader.h" header.

#define LOADER_IGNOREBCASTS  1
#define LOADER_QUIET_BADTYPE 2
#define LOADER_QUIET_BADDROP 4
#define LOADER_QUIET_NOTPERM 8

loader_finalise

const _kernel_oserror *loader_finalise(void);

This function frees all memory and deregisters all event handlers used by the Loader module. You may like to do this before your program exits, though it is not strictly necessary unless your application's heap is in shared memory (e.g. RMA).

All registered Listeners will be systematically removed, along with any message handlers for outstanding dialogues (see loader_deregister_listener). The module's permanent DataLoad, DataSave and DataOpen message handlers will also be deregistered.

To economise on space, this function is not compiled into Loader unless the symbol INCLUDE_FINALISATION_CODE is defined.

Registering listeners

loader_register_listener

const _kernel_oserror *loader_register_listener(
                               unsigned int           flags,
                               int                    file_type,
                               ObjectId               drop_object,
                               const ComponentId     *drop_gadgets,
                               LoaderFileHandler     *loader_method,
                               LoaderFinishedHandler *finished_method,
                               void                  *client_handle);

Call this function to listen for requests to load files of the specified type at the specified location. The arguments are set out below:

Flags word

Only bits 0-3 are currently defined. The flags word should be constructed by bitwise ORing together the C pre-processor symbols defined by the "Loader.h" header.

#define LISTENER_CLAIM       1
#define LISTENER_FILEONLY    2
#define LISTENER_SPRITEAREAS 4
#define LISTENER_FILTER      8

Dropzone and file type

You must specify the area over which the listener will be sensitive to dropped file icons (drop_object, drop_gadgets), and the type of files accepted (file_type). This information will be stored and checked against any DataSave or DataLoad messages received.

Specifying file_type as -1 (defined as constant FILETYPE_ALL) will instruct the listener to accept files of any type. The standard pseudo-types of &1000 for directories, &2000 for application directories and &3000 for untyped files are also valid. An extension in release 15 of CBLibrary allows a pointer to a LoaderPreFilter function to be supplied in place of a file type. This may be more efficient than registering many listeners for the same drop zone to handle different file types.

The argument drop_object should be a Window or Iconbar object id. Specifying this argument as NULL_ObjectId (defined as 0 in "toolbox.h") will make the listener accept all load requests regardless of the window and icon handles specified in the DataSave or DataLoad message.

The argument drop_gadgets may point to an array of ComponentId's, terminated with NULL_ComponentId (defined as -1 in "toolbox.h"). By this mechanism, a single Window object can have several drop zones, and different actions associated with each. Specifying this argument as NULL will make the listener accept all messages for the given Window. It has no defined meaning for Iconbar objects.

Whilst using Wimp window handles and icon numbers would have been more universal, the Toolbox manual states that the icon numbers used for a gadget should not be cached by an application.

Handlers (pre-filter / load / finished)

These are client-supplied routines that will be called when participation in the import process is required.

If flags bit 3 is set then a pointer to a LoaderPreFilter function is expected in place of the file_type argument. This function will be called by the listener to vet requests by other applications for data transfer (including checking the proffered file type).

load_method should be a function that will load data of the specified type from file. If this is specified as NULL then a generic file loader will be used to buffer the raw data in memory. Specifying a LoaderFileHandler forces data received from other applications to be transferred via a temporary file rather than in-memory.

You must supply a finished_method function. This will be called when a data transfer to this listener has been successfully concluded.

The client_handle is not interpreted by the Loader module. It will be passed to the client's LoaderFinishedHandler, and thus may be used to associate a data structure with the object receiving data.

For further details, see the section on handlers.

loader_deregister_listener

const _kernel_oserror *loader_deregister_listener(
                                          int                file_type,
                                          ObjectId           drop_object,
                                          const ComponentId *drop_gadgets);

Deregisters a previously registered listener. Note that the arguments must exactly match those passed to the registration function. Attempting to deregister an unknown listener will result in a failed assertion (or will be ignored, if CBLibrary was compiled with NDEBUG defined).

You should deregister a listener as soon as possible after the deletion of the object for which it was registered (e.g. a window). In practice Loader is tolerant of listeners associated with dead objects or gadgets, to allow for the fact that Toolbox_ObjectDeleted messages are not delivered at very high priority.

Any currently open message dialogues spawned from this listener will be shut down without warning (they cannot exist without their parent listener).

Client-supplied handlers

These are client-supplied routines that allow participation in the process of importing a file. There is currently no support for client participation in RAM transfer.

LoaderPreFilter

typedef loader_pf_result (LoaderPreFilter) (const char *file_path,
                                            int         file_type,
                                            bool        inside_zone,
                                            void       *client_handle);

This routine may be called in response to a DataLoad, DataOpen or DataSave message if your client has specified a pre-filter function instead of a file type. It may even be called if the destination of the incoming data is not within the drop zone specified for this listener (although an argument indicates whether this is the case). Similarly it may be called for cases of direct data transfer regardless of whether LISTENER_FILEONLY was specified upon listener registration.

On entry

file_path will point to a string giving the full canonicalised path of the file being offered to your listener, or NULL if the data being offered is not permanent.

file_type will be the standard RISC OS type of the data (&000-&FFF) or alternatively &1000 for a directory, &2000 for an application directory or &3000 for an untyped file.

If inside_zone is true then the intended destination of the data is one of the gadgets that you specified to loader_register_listener() as the drop zone for this listener. It will also be true if your listener was registered with LISTENER_CLAIM and the Filer is attempting to open an object.

client_handle will be the pointer that you passed to loader_register_listener(). If you follow an object-oriented programming style then this will point to a data structure for the object that registered the listener.

Expected behaviour

In the simplest case your function should simply compare the offered file type against those supported by this listener. It must return value LOADER_PREFILTER_BADTYPE if the file type is unsuitable; Loader will then continue searching for a listener willing to handle the incoming data.

If the file type is valid for this listener but inside_zone is false then your function should return LOADER_PREFILTER_IGNORE; Loader will note that the file type is acceptable and continue searching for an appropriate listener. This mechanism allows it to distinguish between "NoSuchFileType" and "NoSuchDropZone" errors (see 'Errors and the message file').

More complex LoaderPreFilter functions may check the canonical path of the file being offered (if permanent) and return LOADER_PREFILTER_REJECT if the file is already being edited; Loader will cease searching for a willing listener and reject the incoming data transfer. In this circumstance you may also wish to query the user and/or bring the existing editor window to the front.

It is also the job of a LoaderPreFilter function to check whether the data source is permanent and report error "FileNotPerm" if desired; the LISTENER_FILEONLY flag has no effect on listeners that have pre-filters. Your function should return LOADER_PREFILTER_REJECT in this case.

If you wish to accept the incoming data transfer then your function should return LOADER_PREFILTER_CLAIM.

On exit

Your function must return one of the values of enumerated type loader_pf_result. The meaning of these can be summarised as follows:

typedef enum {
  LOADER_PREFILTER_BADTYPE = 0,
  LOADER_PREFILTER_CLAIM,
  LOADER_PREFILTER_REJECT,
  LOADER_PREFILTER_IGNORE
} loader_pf_result;

LoaderFileHandler

typedef Loader2FileHandler LoaderFileHandler;
typedef const _kernel_oserror *(Loader2FileHandler) (const char *file_path,
                                                     flex_ptr    buffer);

This function will be called if your client has specified a custom routine to load data from file. This allows data that must be read directly from file (e.g. by use of a command similar to *Load) to be handled by Loader. Any client-supplied LoaderFileHandler function will not be called when a directory or application directory is received.

In general, you should try to avoid writing code that reads data directly from file, since this prevents Loader from using RAM transfer to receive that type of data. If you instead write code that reads from a buffer (such as the SWI Squash_Decompress) then you can invoke this when your client's LoaderFinishedHandler is called.

On entry

file_path will point to a string giving the full pathname of the file to load. This file may be in !Scrap or it may be a permanent file; it doesn't matter at this stage. buffer will point to a void pointer that initially holds the value NULL (i.e. *buffer == NULL).

Expected behaviour

Your function should determine the buffer size required to hold the entire contents of the specified file. It should call the Flex library function flex_alloc() to allocate this amount of memory, passing the supplied buffer pointer as the anchor for the resultant flex block.

There is no need to explicitly record the buffer size because this can subsequently be read using flex_size() when the same anchor is passed to your LoaderFinishedHandler.

It is good practice to write generalised LoaderFileHandler routines that make no reference to global variables. This allows them to be reused for different listeners and applications. You should not at this stage deal with any of the user-interface aspects of receiving data (apart from anything else, you have not yet been given the intended leafname of the file).

Multi-tasking loading

You must not attempt to do multi-tasking loading by calling any function that invokes SWI Wimp_Poll or Wimp_PollIdle from this function. The resulting delay in answering the data-saving application's messages will be interpreted as a failure to reply, and meanwhile <Wimp$Scrap> may be overwritten by another application.

If you want to implement multi-tasking loading then you could specify LISTENER_FILEONLY when registering the listener, use a dummy LoaderFileHandler, and use the file path passed to your LoaderFinishedHandler. (If you discount data transfer from applications then the title passed to a LoaderFinishedHandler is guaranteed to be a proper file path.)

On exit

Upon returning from a LoaderFileHandler function, *buffer must either be NULL (as on entry) or point to a valid flex memory block.

If your routine could not load the requested file then you should return a pointer to a standard OS error block rather than reporting an error internally. If successful, your routine should return a NULL pointer to indicate that the file has been successfully loaded.

LoaderFinishedHandler

typedef void (LoaderFinishedHandler) (int         drop_x,
                                      int         drop_y,
                                      const char *title,
                                      bool        data_saved,
                                      flex_ptr    buffer,
                                      int         file_type,
                                      void       *client_handle);

This routine will be called when the data transfer protocol has been successfully completed (either in-memory or via a temporary file), and the data has been buffered in memory.

On entry

The drop_x and drop_y arguments are derived from the destination coordinates in the DataSave or DataLoad message originally received from the data-saving task. However, the coordinates passed to the LoaderFinishedHandler will be relative to the work area origin of the destination window. In the case of a file double-clicked in a directory display, these arguments will be -1.

Usually title will point to the canonical file path from which the data was loaded but if the data came directly from another application then it will instead point to the leaf name from the initial DataSave message (or "<untitled>", if this is unknown).

If data_saved is true then the data has been loaded from file rather than transferred directly from another application. You should use this information to mark the document as saved or unsaved, conventionally by appending " *" to the window title if it holds unsaved data.

The buffer argument will point to the anchor of a Flex block used by Loader during the loading process. The data may be accessed as (*buffer)[n] and its length determined by calling flex_size(buffer).

file_type will be the standard RISC OS type of the data (&000-&FFF) or alternatively &1000 for a directory, &2000 for an application directory or &3000 for an untyped file.

client_handle will be the pointer that you passed to loader_register_listener(). If you follow an object-oriented programming style then this will point to a data structure for the object that registered the listener.

Expected behaviour

In this handler you should deal with the loaded data in your own way - open a new document window or dialogue box, or refresh your document display if data has been inserted. You may wish to use the drop coordinates to place the inserted data accurately within a document (unless it has some visible insertion point, such as a caret).

It is your function's responsibility to either call flex_free() to deallocate the flex block referred to by buffer, or reanchor it by calling flex_reanchor(). You must not rely on buffer as a permanent anchor since it will be destroyed upon return from the LoaderFinishedHandler.

If you register a listener with a LoaderFinishedHandler that doesn't make provision for the eventually deallocation of buffer, then your client will leak memory whenever a file is dragged in. Be careful!

Any errors should be reported within your LoaderFinishedHandler. Failure at this stage will not affect the data-saving application, since a DataSaveAck message will already have been sent (after successful load and, if necessary, !Scrap file deletion).

Utility functions

loader_buffer_file

This function is deprecated - you should use loader2_buffer_file() instead.

loader_canonicalise

This function is deprecated - you should use canonicalise() instead.