include.mdp

Include documentation
	Author: Ladislav Mecir

=toc

===Introduction

*Include* is a Rebol script processor. It is implemented as a function and works as a Rebol script loader, preprocessor and evaluator. It is not an alternative module system. In fact, it is orthogonal to such systems, meaning that it can work together with such a system.

===Design features

---A comfortable script evaluator:
* Can preprocess and evaluate any Rebol script (the script does not have to contain *prebol* directives).
* Processes *prebol* directives, facilitating SDK debugging.
* Preprocessing errors are enhanced to inform in which file the error occurred to make debugging more comfortable.
* Uses *include-ctx/path* which resembles paths of many operating systems to search for scripts.
* *include-ctx/path* can contain URLs, not just local directories.
* *include/check* prevents scripts from being re-included (included more than once).
* Processes ordinary scripts with Rebol header.
* Processes prefaced Rebol scripts, i.e., Rebol scripts with preface preceding Rebol header and embedded Rebol scripts, see
	http://www.rebol.com/docs/core23/rebolcore-5.html#section-1.2
* *include/args* sets arguments for the script

---An improved script loader:
* *include/only* loads and preprocesses a script, yielding a Rebol block.
* *include/link* loads a script, preprocesses it and saves the result to the specified file.

---An improved preprocessor:
* Processes all *prebol* directives for basic compatibility with SDK.
* The ~#include-check~ directive is available as a "don't re-include" variant of the ~#include~ directive.
* The ~#paren~ directive is available if the user wants to preprocess a paren.
* The ~comment~ directive is available if the user wants the preprocessor to strip the comments from the script.
* Users can reliably guard parts of their code against preprocessing.
* It is possible to include small parts of scripts like binary data, images or strings.
* Context encapsulation is available using the *make object!* or the *make module!* method.
* Users can define new directives.

===License

Licensed under the Apache License, Version 2.0 (the "License"). You may obtain a copy of the License at

		http://www.apache.org/licenses/LICENSE-2.0

===Description

The preprocessing phase is used to build the script, i.e., to include the necessary parts. Moreover, it enables the user to evaluate any Rebol expression(s).

---Standard preprocessing directives

+++#include script

Include a script (a file or URL) here. The script is preprocessed by *include*.

+++#include-check script

The same as above, except that the script is not included if present in the *include-ctx/log*.

+++#do block

Do the given code block during preprocessing including the result here without preprocessing it further.

+++#paren paren

Preprocess the given paren.

+++#include-string text-file

Include the given *text-file* as a string.

+++#include-binary binary-file

Include the given *binary-file* as a binary.

+++#include-files dir file-list

Include the files from the *file-list* looking for them in the given *dir* directory. The files are included as a block containing the name and the binary contents of all files.

+++comment

Leave out the ~comment~ directive and its argument from the code. The comment is not left out if the ~keep-comments~ global variable is defined when the directive is processed.

---Conditional preprocessing directives

+++#if cond-block true-block

Conditionally include the given *then-block* if the *cond-block* evaluates *true*. The *then-block* is preprocessed when included.

+++#either cond-block true-block false-block

Conditionally include one of the two blocks depending on the result of evaluation of the given *cond-block*. The block being included is preprocessed.

---Global words used

+++include

The ~include~ word refers to the *include* function.

+++include-ctx

The ~include-ctx~ word refers to the object containing important *include*-related variables and helper functions.

+++script?

The implementation patches the *script?* function if needed.

---include-ctx variables

+++path

The ~path~ word refers to a block containing the paths where the *include* function shall look for the files.

If the script file to include is specified with a non-empty path the script file is looked up only at its path. As opposed to that when the script file is specified without any path using only a file name, *include* will use *include-ctx/path* to look for the file.

*include-ctx/path* can be modified/set by users. See also the *include-ctx/push* and *include-ctx/pop* helper functions.

...Prioritization

The directories (or URLs) that are first in *include-ctx/path* are searched first. Consequently they are having priority during the file search.

+++log

The ~log~ variable refers to a block containing the files (with complete paths) *include* function processed. It is used by the ~/check~ refinement and by the ~#include-check~ directive to check whether a specific file was already included. The default Rebol comparison (which is case-insensitive) is used and the paths are converted to lowercase.

The user can save the *log* to a file to create a log file. The user can influence what will be considered as "already included", typically by clearing the contents of the *log* block when starting to link a new file. The files are appended to the *log* block as they are found by *include*, which means that the file found last will be the last one in the log.

+++stack

The stack used by the *push* and *pop* helper functions (see below). There is no need to use the variable directly.

+++block-directive

Handling a subblock or paren.

+++conditional-directives

The definitions of ~#if~ and ~#either~ conditional directives.

+++standard-directives

The definitions of the standard directives (all except the block directive, conditional directives and special includes).

+++special-includes

The definitions of ~#include-binary~, ~#include-string~ and ~#include-files~ directives.

+++directives

The ~directives~ variable refers to the block containing the definitions of directives that are currently used by *include*. There is usually no need to handle this variable directly since *include-ctx/get-directives*, *include-ctx/set-directives* and *include-ctx/update-directives* take care of it.

+++standard-header

The ~standard-header~ variable refers to the script header prototype object. Defined for compatibility with R2 and R3.

+++file

The ~file~ variable refers to the file processed by *include*.

---include-ctx functions

+++include-script

The function used by *include* and by directives to preprocess a script.

+++include-block

The function used by *include-script* and by directives to preprocess a block or a paren. Generated by the *set-directives* function.

+++get-directives

Refers to a function returning the directives block used by *include*. The directives block is the block containing the definitions of all *include* directives.

+++set-directives

Refers to a function setting the directives block used by *include*.

+++update-directives

Refers to a function updating the *directives* (not modifying the block, though) using the given *directives-to-update* block. If a directive contained in the *directives-to-update* block is already present in the *directives* block it is updated. Otherwise it is appended as a new directive.

The usage of the *set-directives* function to update/add new *include directives* is not particularly comfortable. This function should be used as a more comfortable helper.

+++push

Sometimes it is desirable to be able to temporarily redefine *include-ctx/path* while being able to restore its previous value after some time. The *push* function takes a *new-path* argument storing the current *path* contents to the *stack* and using the given *new-path* instead.

+++pop

Restores the previous *path* value from the *stack*. It is the counterpart of the *push* function. If the *stack* is empty, an error is triggered.

+++do-next

Do next expression using the "R3 convention", compatible with R2 and R3.

+++load-next

Load the next value, compatible with R2 and R3.

+++read-binary

Read a binary value from a source, compatible with R2 and R3.

+++make-error

Make the error according to the given specifications, compatible with R2 and R3, allows setting the ~near~ and ~where~ attributes even in R2.

+++disarm-error

Disarm the error, compatible with R2 and R3.

+++redo-error

"enhance" (if needed) and redo the given error.

+++include-error

Create and trigger a new include-type error.

+++split-path

Splits the given path.

+++findpfile

Find the given file using the given search path.

+++find-file

Find a file (using *include-ctx/path* if desired).

---The include error type

A new *include* error type is defined, with the following ids:

+++enhanced

This error represents an, otherwise "normal" error that occurred during code preprocessing. The error is "enhanced" to "know" the file in which it occurred.

+++file-or-url-in-path

A file or a URL was expected in *path*.

+++stack-empty

The *pop* function found the *stack* to be empty.

+++file-not-found

The file to be included was not found.

+++expected

The directive expected an argument.

+++invalid-directive

The *update-directives* function obtained an invalid directive specification in the *directives-to-update* block.

+++script-bug

The *script?* function has a bug in R2. The bug disallows R2 to skip script preface if the *Rebol []* header does not start with capital "R". This error signals that the problem occurred and the script was not loaded correctly.

===Usage examples

A call corresponding to *prebol* processing *%somefile.r* and storing the result of the preprocessing to *%outfile.r* would look like:

    include/link %somefile.r %outfile.r

To change the *include-ctx/path*:

    append include-ctx/path url-or-directory

The *include* call corresponding to the ~#include-check~ directive is *include/check*, which preprocesses and runs the script only if it hasn't done so before:

    include/check %somefile.r

The *include* call using script arguments:

	include/args %somefile.r "my-argument-string"

The distinction between preprocessing directives and "normal" *include* usage is best described on simple examples:

The first example uses *include*, but not the preprocessing directives:

	REBOL [
	    File: %script-builder.r
	]
	; create a new "standard" script from the script base
	verbose: false
	;
	; use a clean LOG
	clear include-ctx/log
	include/link %script-base.r %standard-script.r
	;
	; now create a verbose version
	verbose: true
	clear include-ctx/log
	include/link %script-base.r %verbose-script.r

To create the two versions of the script containing everything they need to run it is sufficient to do:

    include %script-builder.r

which takes care of creating both script versions.

The *%script-base.r* file can look, e.g. as follows:

	REBOL [
	    File: %script-base.r
	]
	#either [verbose] [debug: :print] [debug: none]
	#include-check %part1.r
	#include-check %part2.r
	...

---Context encapsulation

It is possible to encapsulate contexts as follows:

	REBOL [
	    Title: "my script"
	]
	make object! [#include %your-script.r]
	make module! [[module spec goes here] [#include %his-script.r]]

---User-defined directives

New directives are defined using *set-directives* function or, more comfortably, using the *update-directives* function, which allows selective updating of *include directives*. The directives are written in the parse dialect. Even the standard directives are defined this way; it is possible to take their definition as an example.

The *directives* block is bound to the *include-block* function context by the *set-directives* function. Furthermore, the user can bind his *directives* block to *include-ctx* context if his directives use some of the *include-ctx* functions or variables.

The locals of the *include-block* function are:

---pos1

It is used in the definitions of the standard directives to hold the current position in the block (or paren) being included.

This variable can be and frequently is redefined.

---pos2

It is used in the definitions of the standard directives to hold the next position (the position following the currently processed directive part) in the block (or paren) being included.

This variable can be and frequently is redefined.

---value1

In the definitions of the standard directives it is being used to hold the first argument value of the currently processed directive.

This variable can be and frequently is redefined.

---value2

In the definitions of the standard directives it is being used to hold the second argument value of the currently processed directive.

This variable can be and frequently is redefined.

---value3

In the definitions of the standard directives it is being used to hold the third argument value of the currently processed directive.

This variable can be and frequently is redefined.

===The #print directive example

The ~#print~ directive is meant as the debug-print directive. It uses the ~debug~ flag to find out whether to turn the directive into a *print* call, or simply leave it out from the code together with its argument.

Usage:

    #print argument

If the ~debug~ flag (global) is defined (set) when the ~#print~ directive is being processed, the directive is turned into a *print* call; otherwise it is left out from the code.

---A definition of a #print directive

	Rebol [
	    Title: "Debug-print"
	    File: %debug-print.r
	    Author: "Ladislav Mecir"
	    Date: 21-Oct-2011/15:31:41+2:00
	    Purpose: {Defines the #print directive}
	    Notes: {
	        uses the debug flag (global)
	            if the debug flag is set, #print becomes print
	            otherwise, #print and its argument are left out
	    }
	]
	include-ctx/update-directives [
	    #print set value1 skip (
	        if value? 'debug [
	            append linked 'print
	            append/only linked get/any 'value1
	        ]
	    )
	]

===How to's

---Include and relative paths

To be compatible with *do*, *include* changes the working directory to that of the script it is processing. Note that relative paths are relative to the actual working directory, not to the directory that was actual when such paths were put into the *include-ctx/path* block.

For example, if we define:

    include-ctx/path: reduce [%. clean-path %.]

at the first sight it seems that the two elements in the above *include-ctx/path* are equivalent, but that is not true. The first one is a relative path always referring to the current working directory while the second one refers to the directory that was current when the *include-ctx/path* was defined.

---How to evaluate an expression using a #do directive without including a value into the script

This directive evaluates an expression and includes its result 2 into the script:

    #do [1 + 1]

while this one evaluates the expression as well, including nothing:

    #do [1 + 1 []]

---How to write code detecting whether it has been preprocessed

When a script is running, you may need to find out whether it has been preprocessed or not. Here is how:

    ; detect preprocessing
	print case [
		paren? first [#paren ()] [
			"this code was preprocessed by Include"
		]
		integer? first [#do [1]] [
			"this code was preprocessed by prebol"
		]
		'else [
			"this code was not preprocessed"
		]
	]

---How to guard code containing preprocessor directives

The *%include.r* script is an example of a script containing the preprocessor directives that we do not want to be preprocessed.

You may need to handle such situations yourself and write a script so that its behaviour is not influenced by the preprocessor.

	do ([
		; here we can put any code with preprocessor directives
		; without worrying whether the user ran the script using DO or INCLUDE
		; or how many times it was preprocessed
		;
		; this would catch unguarded preprocessing
		#do [do make error! "Unguarded preprocessing."]
		;
		none
	])

---How to run your scripts from Windows Explorer using include

To run your scripts from Windows Explorer using *include* you:

*Download the *%include.r* script.
*Create script files that perform the actions for the Windows Explorer associations.
*Define associations for Windows Explorer context menu.

+++Scripts for Windows Explorer associations

The scripts below implement the actions for the Windows Explorer associations. Do not forget to use your own directory paths and file names.

	Rebol [
		Title: "Halt"
		File: %halt.r
		Author: "Ladislav Mecir"
		Purpose: {
			Includes the first argument file.
			Halts after doint it.
			Works with R2.
		}
	]
	; define the INCLUDE function
	do %include.r
	; include the argument script
	include to-rebol-file system/script/args
	halt

	Rebol [
		Title: "Incl"
		File: %incl.r
		Author: "Ladislav Mecir"
		Purpose: {Includes the first argument file, works with R2}
	]
	; define the INCLUDE function
	do %include.r
	; include the argument script
	include to-rebol-file system/script/args

	Rebol [
		Title: "Incl.r3"
		File: %incl.r3
		Author: "Ladislav Mecir"
		Date: 30-Sep-2011/10:23:03+2:00
		Purpose: {Includes the first argument file, works with R3}
	]
	; define the INCLUDE function
	do %include.r
	;---- R3 GUI
	include %/c/Develop/Rebol/codebase/trunk/r3-gui/loader/loader.r3
	; include the argument script
	include to-rebol-file system/script/args

+++How to define associations for Windows Explorer

You can use an *r_auto_file.reg* file to define the associations I am using. Having added the associations for the *.r* files, left-clicking a *.r* file in the Windows Explorer you will see the new *open*, *halt* and *r3* options, being able to choose which interpreter you want to start. See the example below. Do not forget to edit the source to use your own directory paths and file names.

	Windows Registry Editor Version 5.00

	[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\.r]
	@="r_auto_file"

	[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\r_auto_file]

	[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\r_auto_file\shell]

	[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\r_auto_file\shell\open]

	[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\r_auto_file\shell\open\command]
	@="\"C:\\Program Files\\Notepad++\\Notepad++.exe\" \"%1\""

	[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\r_auto_file\shell\halt.r2]

	[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\r_auto_file\shell\halt.r2\command]
	@="\"C:\\Develop\\Rebol\\rebview.exe\" -s C:\\Develop\\Rebol\\halt.r \"%1\""

	[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\r_auto_file\shell\incl.r2]

	[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\r_auto_file\shell\incl.r2\command]
	@="\"C:\\Develop\\Rebol\\rebview.exe\" -s C:\\Develop\\Rebol\\incl.r \"%1\""

	[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\r_auto_file\shell\incl.r3]

	[HKEY_LOCAL_MACHINE\SOFTWARE\Classes\r_auto_file\shell\incl.r3\command]
	@="\"C:\\Develop\\Rebol\\r3-view.exe\" -s C:\\Develop\\Rebol\\incl.r3 \"%1\""

+++Possible interferences

After using the Windows 7, Windows 8 or Windows 10 built-in "Choose default program..." action (found under the file-right-click context menu under "Open with") it re-associates the extension with whatever new program you choose.
What happens at this point is that *HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Explorer\FileExts\.r\UserChoice* and possibly *HKEY_CURRENT_USER\Software\Microsoft\Windows\Roaming\OpenWith\FileExts\.r\UserChoice* are created/changed by the system and so the newly selected program takes over. To regain control over the extension you can delete the above *UserChoice* key/keys.

===Availability

*Include* is available from *https://github.com/saphirion/include*.

===Changelog

26-Apr-2024/15:25:21+2:00
* update the documentation

20-Apr-2024/9:00:35+2:00
* update the documentation

5-Dec-2020/12:32:03+1:00
* update the documentation

12-Nov-2020/10:54:32+1:00
* add a ~#paren~ directive to allow paren preprocessing when needed
* update the documentation

11-Nov-2020/11:44:26+1:00
* do not allow the preprocessor to process paren contents. Sorry for any discomfort this update may cause to the users, but it has got these advantages:
** makes the behaviour of *include* compatible with *prebol* in this respect
** in contrast to the older versions, this gives the user a reliable tool allowing him to protect code containing preprocessor directives against preprocessing; see the how to
* circumvent a (most likely routine call-related) interpreter bug
* update the documentation

26-Apr-2017/12:47:36+2:00
* add script arguments to *include*
* update the documentation

26-Jul-2015/11:20:13+2:00
* enhance the doc to inform how to revert the "Choose default program..." action effect

5-Dec-2014/9:40:37+1:00

* define the *script-bug* error
* handle the *script?* bug not skipping the script preface when the *Rebol []* header does not start with capital "R"

21-Jun-2013/15:06:50+2:00

* *%include.r* made encappable
* the script and doc moved to Saphirion GitHub

8-Apr-2013/13:00:27+2:00

* *include/link* now using *mold/flat*

15-Mar-2013/12:17:52+1:00

* *include/only* update
** to help Cyphre use it for encapping purposes
** returns the Rebol header spec part in the result block if available

26-Dec-2012/15:00:27+1:00

* license updated to Apache 2.0

14-Nov-2012/15:18:01+1:00

* the ~file~ variable moved from the *include-block* context to *include-ctx*
** scripts can read the variable now,
** "knowing" what is being processed (before, only directives "knew" the variable)
* *include-ctx/load-next* function defined
** for compatibility with R2 and R3
* *script?* function patched
* *include* function:
** header processing reimplemented to really work
** header handling changed
*** it is not a requirement to have a script header now in a manner compatible with R3 *do*
** result corrected in a couple of special cases
** help strings improved
* source punctuation improved
* documentation updated

21-Sep-2012/15:02:28+2:00

* addition of the ~standard-header~ variable to *include-ctx*
* script header processing added
* doc update
* the complete file made available from the rebol.org site

10-Feb-2012/6:55:55+1:00

* adding the *read-binary* function to *include-ctx*
* updating the ~#include-binary~ and ~#include-files~ directives to work in R3

17-Jan-2012/17:57:49+1:00

* circumventing the *append* bug in R2

12-Jan-2012/14:35:27+1:00

* ~/only~ refinement defined for the *update-directives* function
* *special-includes* defined

9-Jan-2012/14:58:59+1:00

* the ~in-file~ bug in standard directives corrected
* simple "pointer" file posted to *http://www.rebol.org/view-script.r?script=ladislav-include.r rebol.org*

10-Nov-2011/10:59:35+1:00

* *include-ctx/standard-directives* subdivided (*block-directive* defined)
* the block directive corrected (the correct ~value1~ variable used)
* a new *invalid-directive* include error defined
* the *update-directives* function updated to trigger the *invalid-directive* error
* *include* type errors documented
* changes documented
* directive identification adjusted
* block-directive adjusted to be identifiable
* ~#if~, ~#either~, ~#do~, ~#include-files~ directives adjusted

8-Nov-2011/15:04:20+1:00

* *include-ctx/conditional-directives* defined (improving the support for localization or other user-defined sets of directives)
* *include-ctx/standard-directives* defined (all standard directives except for the conditional ones)
* comments in the code updated
* no incompatibilities with the previous version expected

21-Oct-2011/23:35:31+2:00

* *update-directives* function adjusted to not modify the *directives* block, replacing it instead (safer)
* *update-directives* function defined
* the *debug-print* example changed to use *update-directives*
* comments added to the code
* the ~#include-binary~ and ~#include-string~ directives adjusted
** handling the situations when the *find-file* function doesn't find the given file
* doc changes

13-Oct-2011/01:58:00+2:00

* RebGUI create-distribution problem solved
* the ~comment~ directive made standard
* the *include-ctx* context defined
* *include* variables (see above) moved to the *include-ctx* context
** (watch out for the obsolete use of ~include-path~, ~include-log~, ~include-push~ and ~include-pop~)
* all helper functions moved to the *include-ctx* context (see above)
** which makes it easier for the user to use any of the helper functions in his own directives
* *include-parser* function does not exist anymore
* *set-directives* function now generates the *include-block* function
* ~localize-block~ removed from the standard code
** (localization code now more complex, but more separate from the "standard code")
* error handling changed
** ~include~ error type defined
** *make-error* function can define the ~near~ and ~where~ error attributes even in R2
* the definition of the ~#print~ directive reflects *include* changes
* documentation updated
* more comments in the *%include.r* file
* *%include.r* code cleaned up

===See also

The article
=url http://www.rebol.org/art-display-article.r?article=w24v
describes how *include* was ported to Rebol 3.