annotate R/rdyncall/vignettes/FLI.Rnw @ 45:da553362fa7c

- forgot to bounce version for last commit on oct 27
author Tassilo Philipp
date Thu, 12 Nov 2020 19:56:33 +0100
parents 0cfcc391201f
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
0
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1 \documentclass[11pt]{article}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
2 \usepackage[round]{natbib}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
3 \usepackage{hyperref}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
4 \usepackage{amsmath}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
5 \usepackage{fancyvrb}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
6 \usepackage{verbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
7 \usepackage{alltt,graphicx}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
8 \usepackage{fullpage}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
9 \bibliographystyle{abbrvnat}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
10 \newcommand{\file}[1]{{`\normalfont\textsf{#1}'}}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
11 \newcommand{\strong}[1]{\texorpdfstring%
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
12 {{\normalfont\fontseries{b}\selectfont #1}}%
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
13 {#1}}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
14 \let\pkg=\strong
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
15 \newcommand\code{\bgroup\@codex}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
16 \def\@codex#1{\texorpdfstring%
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
17 {{\normalfont\ttfamily\hyphenchar\font=-1 #1}}%
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
18 {#1}\egroup}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
19 \newenvironment{smallverbatim}{\small\verbatim}{\endverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
20 \newenvironment{example}{\begin{alltt}}{\end{alltt}}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
21 \newenvironment{smallexample}{\begin{alltt}\small}{\end{alltt}}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
22
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
23 \begin{document}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
24
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
25
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
26 \title{Foreign Library Interface}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
27 %\VignetteIndexEntry{Foreign Library Interface}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
28 \author{by Daniel Adler}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
29 \maketitle
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
30 \abstract{
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
31 We present an improved Foreign Function Interface (FFI) for R to
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
32 call arbitary native functions without the need for C wrapper code.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
33 Further we discuss a dynamic linkage
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
34 framework for binding standard C libraries to R across platforms using a
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
35 universal type information format.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
36 The package \pkg{rdyncall} comprises the framework
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
37 and an initial repository of cross-platform bindings for standard libraries such as
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
38 (legacy and modern) \emph{OpenGL}, the family of \emph{SDL} libraries and \emph{Expat}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
39 The package enables system-level programming using the R language;
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
40 sample applications are given in the article.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
41 We outline the underlying automation tool-chain that extracts
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
42 cross-platform bindings from C headers, making the
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
43 repository extendable and open for library developers.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
44 }
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
45 \section{Introduction}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
46
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
47 \begin{table*}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
48 \centering
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
49 \label{tab:libs}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
50 \begin{tabular}{l|l|c|c|c}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
51 lib/dynport & description & functions & constants & aggregate types \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
52 \hline
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
53 \code{gl} & opengl & 337 & 3253 & - \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
54 \code{glu} & opengl utility & 59 & 154 & - \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
55 \code{r} & r library & 238 & 700 & 27 \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
56 \code{sdl} & audio/video/ui abstraction & 203 & 465 & 51 \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
57 \code{sdl\_image} & pixel format loaders & 29 & - & - \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
58 \code{sdl\_mixer} & music format loaders and playing & 63 & 12 & - \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
59 \code{sdl\_ttf} & font format loaders & 35 & 9 & - \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
60 \code{cuda} & gpu programming & 387 & 665 & 84 \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
61 \code{expat} & xml parsing framework & 65 & 70 & - \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
62 \code{glew} & gl extensions & 1465 & - & - \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
63 \code{gl3} & opengl 3 (strict) & 324 & 838 & 1 \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
64 \code{opencl} & gpu programming & 78 & 260 & 10 \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
65 \code{stdio} & standard i/o & 76 & 3 & - \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
66 \end{tabular}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
67 \caption{overview of available dynports for portable c libraries}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
68 \end{table*}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
69
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
70 We present an improved Foreign Function Interface (FFI) for R that
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
71 significantly reduces the amount of C wrapper code needed to interface with C.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
72 We also introduce a \emph{dynamic} linkage that binds the C
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
73 interface of a pre-compiled library (\emph{as a whole}) to an
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
74 interpreted programming environment \citep{Oust97a} such as R - hence th name
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
75 \emph{Foreign Library Interface}. Table 1 gives a list
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
76 of the C libraries currently supported across major R platforms.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
77 For each library supported, abstract interface specifications are declared
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
78 in a compact platform-neutral text-based format stored in so-called
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
79 \emph{DynPort} files on a local repository.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
80
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
81 %between high-level interpreted programming environments
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
82 %and native pre-compiled C libraries that uses a compact text-based
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
83 %interface and type information format that makes this method work across platforms.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
84
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
85 R \citep{R:Ihaka+Gentleman:1996} was choosen as the first language
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
86 to implement a proof-of-concept implementation for this approach.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
87 This article describes the \pkg{rdyncall} package which
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
88 implements a complete toolkit of low-level facilities that can be used as an
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
89 alternative FFI to interface with the C programming language.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
90 And further, it enables direct and quick access to
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
91 the common C libraries from R without compilation.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
92
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
93 The project was motivated by the fact that
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
94 high-quality software solutions implemented in portable C
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
95 are often not available in interpreter-based languages such as R.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
96 The pool of freely available C libraries is quite large and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
97 represents an invaluable resource for software development.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
98 For example, OpenGL \citep{Board05} is the most portable and standard interface to
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
99 accelerated graphics hardware for developing real-time graphics software.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
100 The combination of OpenGL with the \emph{Simple DirectMedia Layer} (SDL) \citep{SDL}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
101 core and extension libraries offers a foundation framework for
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
102 developing interactive multimedia applications that can run on a
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
103 multitude of platforms.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
104 Other libraries such as the Expat XML Parser \citep{www:expat} provide a parser framework
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
105 for processing very large XML documents.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
106 And even the C library of R contains high-quality statistical
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
107 functions that are useful in context of other languages as well.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
108
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
109 To make use of these libraries within high-level languages, \emph{language bindings}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
110 to the library must be written as an extension to the language, a task that
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
111 requires deep familiarity of the internals of both the library and the interpreter.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
112 Depending on the complexity of the library, the amount of work needed to wrap
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
113 the interface can be very large (Table \ref{tab:libs} gives the counts of
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
114 functions, constants and types that need to be wrapped).
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
115 Rather than having to write a separate binding for each \emph{library and language}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
116 combination, we research a dynamic binding approach that
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
117 is adaptable to interpreters and works cross-platform without additional
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
118 compilation of wrapper layers.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
119 Once the binding specification for a library has been specified, that
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
120 library becomes automatically accessible to all interpreters that
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
121 implement such a framework outlined here.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
122 Extension techniques offered by the language interpreter, such as a
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
123 \emph{Foreign Function Interface} (FFI), are the fundamental technology
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
124 for bridging the dynamic interpreter with statically pre-compiled code.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
125
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
126 In the case of R the built-in FFI function \code{.C} provides a fairly
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
127 basic call gate to C code with strong limitations; additional wrapper code has
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
128 to be written in addition to interface with standard C libraries.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
129 \pkg{rdyncall} contributes an improved FFI for R that offers a \emph{flexible}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
130 and \emph{type-safe} interface with support for almost all C types without
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
131 requiring additional C wrappers.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
132
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
133 Based on this FFI, the package contains a proof-of-concept implementation of a \emph{Foreign Library Interface} that enables
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
134 \emph{direct} and \emph{dynamic} interoperability with foreign C Libraries
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
135 (including shared library code and the Application Programming Interface
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
136 specified in C headers) from within the R interpreter.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
137 For each C library supported, abstract interface specification are declared in a
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
138 compact platform-neutral text-based format stored in a so-called \emph{DynPort} file
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
139 located in a local repository within the package.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
140 Table \ref{tab:libs} gives a sample list of available bindings that come with the package.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
141
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
142 Users gain access to C libraries from R using the front-end function \code{dynport(}\emph{portname}\code{)},
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
143 which processes a \emph{DynPort} file to load the C library\footnote{Pre-compiled libraries need to be installed, OS-specific installation notes are given in the documentation of the package.},
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
144 and wrap the C interface as a newly attached R environment
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
145 \footnote{Note \pkg{rdyncall} version 0.7.4 and below uses R name space objects \citep{RNameSpace} as dynport containers. This has changed starting with version 0.7.5 due to restrictions for packages hosted on CRAN not to use internal functions. Since there is no public interface for the creation of name space objects currently in R, \pkg{rdyncall} uses ordinary environment objects for now.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
146 This disables the use of the double colon operator (\code{::}) to refer to dynport objects; unloading is done using \code{detach(dynport:<PORTNAME>)}.}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
147 that uses the same symbolic names of the C API.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
148 R code that uses C interfaces via \emph{DynPort}s might look very familiar to C user code.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
149
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
150 This article motivates the topic with a comparison of the built-in and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
151 contributed FFI by means of a simple use case. This leads to a detailed description of the improved FFI.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
152 Then follows an overview of the package and a brief tour through the framework
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
153 with details on the handling of foreign C data types and wrapping R functions as callbacks.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
154 Two sample applications are given using OpenGL, SDL and Expat.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
155 The article ends with a brief description of the implementation based on C libraries from the \emph{DynCall} project \citep{dyncall}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
156 and the tool-chain that was used to create the repository of \emph{DynPort} files.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
157
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
158 \section{Foreign Function Interfaces}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
159
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
160 FFIs provide the backbone of a language to interface with foreign code.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
161 Depending on the design of this service,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
162 it can largely unburden developers from writing additional wrapper code.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
163 In this section, we compare the built-in FFI with the improved
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
164 FFI provided by \pkg{rdyncall} using a simple example that sketches
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
165 the different work flow paths for making an R binding to a function
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
166 from a foreign C library.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
167
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
168 \subsection{FFI of base R}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
169
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
170 Suppose that we wish to invoke the C function \code{sqrt} of the
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
171 C Standard Math library. The function is declared as follows in C:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
172 \begin{verbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
173 double sqrt(double x);
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
174 \end{verbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
175
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
176 R offers a number of functions to call pre-compiled code from
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
177 within the R interpreter. While \code{.Call} and \code{.External}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
178 are designed for interoperability with \emph{extension} code, \code{.C}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
179 and \code{.Fortran} seem to offer the most low-level interoperability with
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
180 \emph{foreign} code.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
181 But \code{.C} has also very strict conversion rules and strong limitations
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
182 regarding argument and return-types:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
183 \code{.C} passes R arguments as C pointers and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
184 C return types are not supported, so only C \code{void} functions,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
185 which are procedures, can be called.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
186 Given these limitations, we are not able to invoke the foreign
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
187 \code{sqrt} function directly and need some intermediate wrapper code
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
188 written in C that obeys the rules of the \code{.C} interface:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
189
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
190 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
191 #include <math.h>
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
192 void R_C_sqrt(double * ptr_to_x)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
193 {
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
194 double x = ptr_to_x[0], ans;
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
195 ans = sqrt(x);
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
196 ptr_to_x[0] = ans;
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
197 }
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
198 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
199
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
200
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
201 We assume that the wrapper code is deployed as a shared library
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
202 in a package named \emph{testsqrt} which links to the C math library.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
203 \footnote{We omit here the details such as registering C functions which is
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
204 described in detail in the R Manual '\emph{Writing R Extensions}' \citep{RExt}.}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
205 Then we load the \emph{testsqrt} package and call the C wrapper function directly
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
206 via \code{.C}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
207
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
208 \begin{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
209 > library(testsqrt)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
210 > .C("R_C_sqrt", 144, PACKAGE="testsqrt")
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
211 [[1]]
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
212 [1] 12
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
213 \end{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
214
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
215 To make \code{sqrt} available as a public function, an additional
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
216 R wrapper layer is added, that does type-safety checks before
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
217 issuing the \code{.C} call.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
218
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
219 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
220 sqrtViaC <- function(x)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
221 {
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
222 x <- as.numeric(x) # type(x) should be C double.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
223 # make sure length > 0:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
224 length(x) <- max(1, length(x))
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
225 .C("R_C_sqrt", x, PACKAGE="example")
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
226 }
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
227 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
228
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
229 As an alternative, R also provides high-level C extension interfaces
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
230 such as \code{.Call} and \code{.External}, that give access to R internals
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
231 at C level and enable to make type-safety checks within C:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
232
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
233 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
234 #include <R.h>
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
235 #include <Rinternals.h>
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
236 #include <math.h>
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
237 SEXP R_Call_sqrt(SEXP x)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
238 {
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
239 SEXP ans = R_NilValue, tmp;
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
240 PROTECT( tmp = coerceVector(x, REALSXP) );
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
241 if (LENGTH(tmp) > 0) {
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
242 double y = REAL(tmp)[0], result;
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
243 result = sqrt(y);
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
244 ans = ScalarReal(result);
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
245 }
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
246 UNPROTECT(1);
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
247 return ans;
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
248 }
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
249 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
250
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
251 Now the corresponding R wrapper shrinks into a simple delegate:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
252
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
253 \begin{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
254 > sqrtViaCall <- function(x)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
255 + .Call("R_Call_sqrt", x, PACKAGE="example")
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
256 \end{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
257
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
258 The third alternative, via \code{.External}, is omitted here;
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
259 it has a different argument passing scheme, but the C and R wrapper
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
260 implementations would look very similar.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
261
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
262 We can conclude that - in realistic settings - the built-in FFI of R
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
263 almost always needs support by a wrapper layer written in C.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
264 The "foreign" in FFI is in fact relegated to the C wrapper layer.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
265
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
266 Moreover the R FFI can be viewed as an \emph{extension} interface for
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
267 calling pre-compiled code written in a \emph{foreign} language within
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
268 the context of the R implementation, rather than a direct invocation
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
269 interface for code from a \emph{foreign} context such as an
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
270 ordinary C library.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
271
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
272 \subsection{FFI of rdyncall}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
273
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
274 \begin{table*}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
275 \begin{center}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
276 \begin{tabular}{ll|ll}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
277 \hline \hline
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
278 Type& Sign. & Type & Sign. \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
279 \hline
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
280 \verb@void@ & \verb@v@ & \verb@bool@ & \verb@B@ \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
281 \verb@char@ & \verb@c@ & \verb@unsigned char@ & \verb@C@ \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
282 \verb@short@ & \verb@s@ & \verb@unsigned short@ & \verb@S@ \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
283 \verb@int@ & \verb@i@ & \verb@unsigned int@ & \verb@I@ \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
284 \verb@long@ & \verb@j@ & \verb@unsigned long@ & \verb@J@ \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
285 \verb@long long@ & \verb@l@ & \verb@unsigned long long@ & \verb@L@ \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
286 \verb@float@ & \verb@f@ & \verb@double@ & \verb@d@ \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
287 \verb@void*@ & \verb@p@ & \verb@struct@ \emph{name} \verb@*@ & \verb@*<@\emph{name}\verb@>@ \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
288 \emph{type}\verb@*@ & \verb@*@... & \verb@const char*@ & \verb@Z@ \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
289 \hline \hline
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
290 \end{tabular}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
291 \end{center}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
292 \caption{\label{tab:signature} C/C++ Types and Signatures}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
293 \end{table*}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
294
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
295 \pkg{rdyncall} provides an improved FFI for R
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
296 that is accessible via the function \code{.dyncall}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
297 In contrast to the built-in R FFI which uses a C wrapper layer,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
298 the \code{sqrt} function is invoked dynamically and directly
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
299 by the interpreter at run-time.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
300 Whereas the C math library was loaded implicitly via the
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
301 example package, it now has to be loaded explicitly.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
302
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
303 R offers functions to deal with shared libraries at run-time,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
304 but the location has to be specified as an absolute pathname which
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
305 is platform-specific.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
306 For now, let us assume that the example is done on
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
307 Mac OS X where the C math library is located
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
308 at \file{/usr/lib/libm.dylib}. A platform-portable solution
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
309 is discussed in the next section on \emph{Portable loading of shared library}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
310
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
311 \begin{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
312 > libm <- dyn.load("/usr/lib/libm.dylib")
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
313 > sqrtAddr <- libm$sqrt$address
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
314 \end{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
315
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
316 We first need to load the R package \pkg{rdyncall}:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
317
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
318 \begin{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
319 > library(rdyncall)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
320 \end{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
321
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
322 Finally, we invoke the foreign C function \code{sqrt} \emph{directly} via
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
323 \code{.dyncall}:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
324
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
325 \begin{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
326 > .dyncall(sqrtAddr, "d)d", 144)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
327 [1] 12
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
328 \end{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
329
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
330 Let us review the last call, as it pinpoints the core solution for a direct
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
331 invocation of foreign code within R:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
332 The first argument specifies the address of the foreign code, given as an
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
333 external pointer.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
334 The second argument is a \emph{call signature}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
335 that specifies the argument- and return types of the target C function.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
336 This string \verb@"d)d"@ specifies that the foreign function
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
337 expects a \code{double} scalar argument and returns a \code{double} scalar value
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
338 in correspondence to the C declaration of \code{sqrt}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
339 Arguments following the call signature are passed to the
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
340 foreign function using the call signature for type-safe conversion to C types.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
341 In this case we pass \code{144} as a C \code{double} argument type as first
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
342 argument and receive a C \code{double} value converted to an R \code{numeric}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
343
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
344 \subsection{Call Signatures}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
345
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
346 The introduction of a type descriptor for foreign functions is a key
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
347 component that makes the FFI flexible and type-safe.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
348 The format of the call signature has the following pattern:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
349
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
350 \begin{center}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
351 \emph{argument-types} \verb@')'@ \emph{return-type}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
352 \end{center}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
353
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
354 The signature can be derived from the C function declaration:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
355 Argument types are specified first, in a left-to-right order, and are
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
356 terminated by the \verb@')'@ symbol followed by a single return type signature.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
357
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
358 Almost all fundamental C types are supported and there is no real
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
359 restriction regarding the number of arguments supported to issue
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
360 a call.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
361 Table \ref{tab:signature} gives an overview of supported C types and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
362 the corresponding text encoding; Table \ref{tab:signature_examples}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
363 provides some examples of C functions and call signatures.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
364
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
365 \begin{table*}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
366 \center
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
367 \begin{tabular}{l|l}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
368 C function declaration & dyncall type signature \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
369 \hline
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
370 \verb@void rsort_with_index(double*,int*,int n)@ & \verb@*d*ii)v@ \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
371 \verb@SDL_Surface * SDL_SetVideoMode(int,int,int,Uint32_t)@ & \verb@iiiI)*<SDL_Surface>@ \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
372 \verb@void glClear(GLfloat,GLfloat,GLfloat,GLfloat)@ & \verb@ffff)v@ \\
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
373 \end{tabular}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
374 \caption{\label{tab:signature_examples}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
375 Some examples of C functions and corresponding signatures}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
376 \end{table*}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
377
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
378 Now, let us define a public and type-safe R wrapper function that
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
379 hides the details of the foreign function call by passing the formal
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
380 argument place holder "\code{...}" as third argument to \code{.dyncall}:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
381
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
382 \begin{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
383 > sqrtViaDynCall <- function(...)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
384 + .dyncall(sqrtAddress, "d)d", ...)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
385 \end{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
386
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
387 Although there is no further guard code, this interface is type-safe and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
388 the user can do no harm by inadvertently using a wrong set and/or type
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
389 of arguments due to the built-in type-checks.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
390 Compared to the R wrapper code using \code{.C}, no explicit cast of the
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
391 arguments via \code{as.numeric} is required, because
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
392 automatic coercion rules for fundamental types are implemented as dictated
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
393 by the call signature. For example, \code{integer} R values are
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
394 implicitly casted to \code{double} automatically:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
395
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
396 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
397 > sqrtViaDyncall(144L)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
398 [1] 12
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
399 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
400
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
401 A certain level of type-safety is achieved here as well:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
402 All arguments to be passed to C are first checked against the call signature.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
403 If any incompatibility is detected, such as a wrong number of arguments,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
404 empty atomic vectors or incompatible type mappings, the invocation is aborted
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
405 and an error is reported before risking an application crash:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
406
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
407 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
408 > sqrtViaDyncall(1,2)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
409 Error in .dyncall(sqrtAddress, "d)d", ...) :
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
410 Too many arguments for signature 'd)d'.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
411 > sqrtViaDyncall()
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
412 Error in .dyncall(sqrtAddress, "d)d", ...) :
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
413 Not enough arguments
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
414 for function-call signature 'd)d'.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
415 > sqrtViaDyncall(NULL)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
416 Error in .dyncall(sqrtAddress, "d)d", ...) :
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
417 Argument type mismatch at position 1:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
418 expected double convertible value
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
419 > sqrtViaDyncall("144")
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
420 Error in .dyncall(sqrtAddress, "d)d", ...) :
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
421 Argument type mismatch at position 1:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
422 expected double convertible value
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
423 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
424
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
425 In contrast to the R FFI, where the argument conversion is
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
426 dictated solely by the R argument type at call-time in a one-way fashion,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
427 the introduction of an additional specification with a call signature gives
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
428 several advantages.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
429
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
430 \begin{itemize}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
431 \item Almost all possible C functions can be invoked by a single interface;
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
432 no additional C wrapper is required.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
433 \item The built-in type-safety checks of passed arguments enhance stability
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
434 and reduce assertion code in R wrappers significantly.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
435 \item A single call signature can work across platforms,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
436 given that the C function type remains constant across platforms.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
437 \item Given that our FFI is implemented in multiple languages,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
438 call signatures represent a portable type description for C libraries.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
439 \end{itemize}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
440
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
441 \section{Package Overview}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
442
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
443 Besides dynamic calling of foreign code, the package provides essential
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
444 facilities for interoperability between the R and C programming languages.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
445 A high-level overview of components that make up the
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
446 package is given in Figure \ref{fig:pkg_overview}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
447
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
448 \begin{figure}[h]
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
449 \centering
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
450 \includegraphics[scale=0.44]{img_overview.pdf}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
451 \caption{\label{fig:pkg_overview}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
452 Package Overview}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
453 \end{figure}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
454
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
455 We already described the \code{.dyncall} FFI. It follows a
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
456 brief description of portable loading of
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
457 shared libraries using \code{dynfind}, installation of wrappers via \code{dynbind},
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
458 handling of foreign data types via \code{new.struct} and wrapping of R functions as C callbacks via \code{new.callback}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
459 Finally the high-level \code{dynport} interface for accessing \emph{whole} C libraries is briefly discussed.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
460 The technical details at low-level of some components are described briefly in the
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
461 section \emph{Architecture}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
462
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
463 \subsection{Portable loading of shared libraries}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
464
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
465 The \emph{portable} loading of shared libraries across platforms is not
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
466 trivial because the file path is different in Operating-Systems (OS).
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
467 Referring back to the previous example, to load a particular library
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
468 in a portable fashion, one would have to check the platform to
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
469 locate the C library.\footnote{Possible C math library names are \file{libm.so}, \file{libm.so.6} and \file{MSVCRT.DLL}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
470 in locations such as \file{/lib}, \file{/usr/lib}, \file{/lib64}, \file{/lib/sparcv9}, \file{/usr/lib64}, \file{C:\textbackslash WINDOWS\textbackslash SYSTEM32} etc..}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
471
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
472 Although there is variation among the OSs, library file paths and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
473 search patterns have common structures.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
474 For example, among
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
475 all the different locations, prefixes and suffixes, there is a part within
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
476 a full library filename that can be taken as a \emph{short library name} or
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
477 label.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
478
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
479 The function \code{dynfind} takes a list of short library names to
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
480 locate a library using common search heuristics.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
481 For example, to load the Standard C Math library, one would either use
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
482 the Microsoft Visual C Run-Time library labeled \file{msvcrt} on Windows
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
483 or the C Math library labeled \file{m} or \file{m.so.6} otherwise.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
484
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
485 \begin{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
486 > mLib <- dynfind(c("msvcrt","m","m.so.6"))
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
487 \end{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
488
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
489 \code{dynfind} also supports more exotic schemes, such as the Mac OS X Framework folders.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
490 Depending on the library,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
491 it is sometimes enough to have a single short filename - e.g. \code{"expat"} for
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
492 the \emph{Expat} library.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
493
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
494 Internally, the dynamic linker interface of the OS is used via
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
495 \code{.dynload} and symbols get resolved via \code{.dynsym}:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
496
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
497 \begin{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
498 > sqrtAddr <- .dynsym(mLib, "sqrt")
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
499 \end{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
500
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
501 Although R already contains support for loading shared libraries
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
502 and resolving of symbols, several issues have led to a reimplementation
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
503 of this part:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
504
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
505 \begin{itemize}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
506 \item System paths are not considered when loading libraries via
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
507 \code{dyn.load} of the package \pkg{base} but this is one part of the
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
508 search heuristics.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
509 \item Automatic life-cycle management for loading and unloading of libraries
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
510 is a desired goal. Unloading of libraries should be done automatically
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
511 via finalizer code when no symbols are used anymore. External pointers
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
512 resolved via \code{.dynsym} hold a reference to the loaded library.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
513 When all external pointers are garbage collected, the library handle is
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
514 not referenced anymore and the finalizer can unload the library.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
515 \end{itemize}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
516
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
517 \subsection{Wrapping C libraries}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
518
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
519 Functional R interfaces to foreign code can be defined with small
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
520 R wrapper functions, which effectively delegates to \code{.dyncall}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
521 Each function interface is parameterized by a target address and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
522 a matching call signature.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
523
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
524 Since APIs often consist of hundreds of functions (see Table \ref{tab:libs}),
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
525 \code{dynbind} can create and install a batch of function wrappers for a library
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
526 with a single call by using a \emph{library signature} that
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
527 consists of concatenated function names and signatures separated by semicolons.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
528
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
529 For example, to install wrappers to the C functions
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
530 \code{sqrt}, \code{sin} and \code{cos} from the math library, one
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
531 could use:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
532
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
533 \begin{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
534 > dynbind( c("msvcrt","m","m.so.6"),
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
535 + "sqrt(d)d;sin(d)d);cos(d)d;" )
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
536 \end{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
537
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
538 The function call has the side-effect that three R wrapper functions are
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
539 created and stored in an environment which defaults to the global environment.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
540 Let us review the \code{sin} wrapper (on the 64-bit Version of R running
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
541 on Mac OS X 10.6):
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
542 \begin{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
543 > sin
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
544 function (...)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
545 .dyncall.default(<pointer: 0x7fff81fd13f0>,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
546 "d)d)", ...)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
547 \end{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
548
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
549 The wrapper directly uses the address of the resolved \code{sin} symbol.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
550 In addition, the wrappers uses \code{.dyncall.default}, which is a
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
551 concrete selector of a particular calling convention, as outlined below.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
552
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
553 \subsection{Calling Conventions}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
554
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
555 Calling conventions specify how arguments and return values are passed
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
556 across sub-routines and functions at machine level. This information
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
557 is vital for interfacing with the binary interface of C libraries.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
558 The package has support for multiple calling conventions.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
559 Calling conventions are controlled by \code{.dyncall} via the named argument
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
560 \code{callmode} to specify a non-default calling convention.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
561 Most current OSs and platforms only have support for a single \code{"default"} calling convention
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
562 at run-time.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
563
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
564 An important exception is the Microsoft Windows platform
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
565 on the 32-bit \emph{i386} processor architecture:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
566 While the default C calling convention on \emph{i386} is \code{"cdecl"} (which is the \code{"default"} on \emph{i386}),
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
567 system shared libraries from Microsoft such as \file{KERNEL32.DLL},
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
568 \file{USER32.DLL} and the OpenGL library \file{OPENGL32.DLL}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
569 use the \code{"stdcall"} calling convention.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
570 Only on this platform, the \code{callmode} argument has an effect and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
571 selects the calling convention to be used when working on Microsoft Windows 32-Bit.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
572 All other platforms currently ignore this argument.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
573
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
574 \subsection{Handling of C Types in R}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
575
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
576 C APIs often make use of high-level C \verb@struct@
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
577 and \verb@union@ types for exchanging information.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
578 Thus, to make interoperability work at that level the handling of C
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
579 type information is addressed by the package.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
580
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
581 Let us consider the following hypothetical example:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
582 A user-interface library has a function to set the 2D coordinates
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
583 and dimension of a graphical output window. The coordinates are specified using a C
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
584 \code{struct Rect} data type and the C function receives a
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
585 pointer on that object:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
586
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
587 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
588 void setWindowRect(struct Rect *pRect);
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
589 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
590
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
591 The structure type is defined as follows:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
592
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
593 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
594 struct Rect {
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
595 short x, y;
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
596 unsigned short w, h;
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
597 };
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
598 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
599
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
600 Before we can issue a call, we have to allocate an object of that size and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
601 initialize the fields with values encoded in C types, which are not
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
602 part of R data types.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
603 The framework provides helper functions and objects to deal with C data types
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
604 in R. Type information objects can be created with a description of the
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
605 C aggregate structure.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
606 First, we create a type information object in R for the \code{struct Rect}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
607 C data type via \code{parseStructInfos} using a \emph{structure type signature}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
608
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
609 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
610 > parserStructInfos("Rect{ssSS}x y w h;")
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
611 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
612
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
613 After registration, an R object named \code{Rect} is installed, which
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
614 contains C type information that corresponds to \code{struct Rect}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
615 The format of a \emph{structure type signature} has the following
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
616 pattern:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
617
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
618 \begin{center}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
619 \emph{Struct-name} \verb@'{'@ \emph{Field-types} \verb@'}'@ \emph{Field-names} \verb@';'@
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
620 \end{center}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
621
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
622 \emph{Field-types} use the same type signature encoding as that of
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
623 \emph{call signatures} for argument and return types (Table \ref{tab:signature}).
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
624 \emph{Field-names} consist of a list of white-space separated names,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
625 labeling each field component.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
626
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
627 An instance of a C type can be allocated via \code{new.struct}:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
628
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
629 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
630 > r <- new.struct(Rect)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
631 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
632
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
633 Finally, the extraction (\verb@'$'@, \verb@'['@) and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
634 replacement(\verb@'$<-'@, \verb@'[<-'@) operators can be used to access
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
635 structure fields symbolically. During value transfer between R and C,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
636 automatic conversion of values with respect to the underlying C field
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
637 type takes place.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
638
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
639 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
640 > r$x <- -10 ; r$y <- -20 ; r$w <- 40 ; r$h <- 30
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
641 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
642
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
643 In this example, R \code{numeric} values are converted on the fly to \code{signed}- and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
644 \code{unsigned short} integers (usually 16-bit values). When the object gets printed on the prompt,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
645 a detailed picture of the data object is given:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
646
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
647 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
648 > r
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
649 struct Rect {
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
650 x: -10
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
651 y: -20
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
652 w: 40
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
653 h: 30
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
654 }
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
655 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
656
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
657 At low-level, one can see that \code{r} is stored as an R \code{raw} vector object:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
658
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
659 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
660 > r[]
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
661 [1] f6 ff ec ff 28 00 1e 00
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
662 attr(,"struct")
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
663 [1] "Rect"
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
664 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
665
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
666 To follow the example, we issue a foreign function call to \code{setRect}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
667 via \code{.dyncall} and pass in the \code{r} object,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
668 assuming the library is loaded and the symbol is resolved and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
669 stored in an external pointer object named \code{setWindowRectAddr}:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
670
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
671 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
672 > .dyncall( setWindowRectAddr, "*<Rect>)v", r)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
673 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
674
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
675 We make use of a typed pointer expression \code{'*<Rect>'}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
676 instead of the untyped pointer signature \code{'p'}, which would
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
677 also work but does not prevent users from passing other objects
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
678 that do not reference a \code{struct Rect} data object.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
679 Typed pointer expressions increase type-safety and use the
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
680 pattern \verb@'*<@\emph{Type-Name}\verb@>'@.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
681 The invocation will be rejected if the argument passed in is not
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
682 of C type \code{Rect}. As \code{r} is tagged with an attribute
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
683 \code{struct} that refers to \code{Rect}, the call will be issued.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
684
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
685 Typed pointers can also occur as return types that - once the
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
686 type information is available - permit the manipulation of returned objects
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
687 in the same symbolic manner as above.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
688
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
689 C \verb@union@ types are supported as well but use the \code{parseUnionInfos}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
690 function instead for registration and a slightly different signature format:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
691
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
692 \begin{center}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
693 \emph{Union-name} \verb@'|'@ \emph{Field-types} \verb@'}'@ \emph{Field-names} \verb@';'@
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
694 \end{center}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
695
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
696 The underlying low-level C type read- and write operations and conversions
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
697 from R data types are performed by the functions \code{.pack} and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
698 \code{.unpack}. These can be used for various low-level operations as well,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
699 such as dereferencing of pointers on pointers.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
700
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
701 R objects such as external pointers and atomic raw, integer and numeric
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
702 vectors can be used as aggregate C types via the attribute \code{struct}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
703 To \emph{cast} a type in the style of C, one can use \code{as.struct}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
704
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
705 \subsection{Wrapping R functions as C callbacks}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
706
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
707 Some C libraries, such as user-interface toolkits and I/O processing
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
708 frameworks, use \emph{callbacks} as part of their interface to enable
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
709 registration and activation of user-supplied event handlers.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
710 A callback is a user-defined function that has a library-defined
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
711 function type. Call-backs are usually registered via a registration function
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
712 offered by the library interface and are activated later from within
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
713 a library run-time context.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
714
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
715 \pkg{rdyncall} has support for wrapping ordinary R
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
716 functions as C callbacks via the function
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
717 \code{new.callback}. Callback wrappers are defined by a \emph{callback
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
718 signature} and the user-supplied R function to be wrapped. \emph{Callback signatures} look very
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
719 similar to \emph{call signatures} and should match the
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
720 functional type of the underlying C callback.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
721 \code{new.callback} returns an external pointer that can
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
722 be used as a low-level function pointer for the registration as a C callback.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
723 See Section \emph{Parsing XML using Expat} below for
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
724 applications of callback.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
725
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
726 \subsection{Foreign Library Interface}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
727
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
728 At the highest level, \pkg{rdyncall} provides the front-end function
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
729 \code{dynport} to dynamically setup an interface to a C Application
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
730 Programming Interface. This includes loading of the corresponding
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
731 shared C library and resolving of symbols. During the binding process,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
732 a new R environment (this was a name space \citep{RNameSpace} till version 0.7.4) will be populated with thin R wrapper
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
733 objects that represent abstractions to C counter-parts such as
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
734 functions, pointer-to-functions, type-information objects for C struct and union
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
735 types and symbolic constant equivalents of C enums and macro defines.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
736 The mechanism aims to work across platforms, given that the corresponding
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
737 shared libraries of a \emph{DynPort} have been installed in a
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
738 system standard location on the host.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
739
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
740 An initial repository of \emph{DynPorts} is available in the package
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
741 that provides bindings for several popular C APIs, see Table \ref{tab:libs}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
742 for examples of available bindings.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
743
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
744 \section{Sample Applications}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
745
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
746 We give two examples with different application contexts that demonstrate
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
747 the direct usage of C APIs from within R through the \pkg{rdyncall} package.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
748 The R interface to C libraries looks very
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
749 similar to the actual C API. For details on the usage of a particular
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
750 C library, the programming manuals and documentation of the libraries
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
751 should be consulted.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
752
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
753 Before loading R bindings via \code{dynport}, the shared library should
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
754 have been installed onto the system. Currently this is
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
755 to be done manually and the installation method depends on the target OS (See the manual
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
756 page about the 'rdyncall-demos' for details on this).
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
757 While \emph{OpenGL} is most often pre-installed on typical desktop-systems,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
758 \emph{SDL} and \emph{Expat} sometimes have to be installed explicitly.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
759
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
760 \subsection{OpenGL Programming in R}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
761
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
762
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
763 In the first example, we make use of the Simple DirectMedia Layer library (SDL)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
764 \citep{SDL} \citep{Pendleton:2003:GPS} \citep{www:sdl-alternative} and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
765 the Open Graphics Library (OpenGL) \citep{Board05} to implement
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
766 a portable multimedia application skeleton in R.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
767
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
768 We first need to load bindings to SDL and OpenGL via dynports:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
769
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
770 \begin{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
771 > dynport(SDL)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
772 > dynport(GL)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
773 \end{example}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
774
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
775 Now we initialize the SDL library - in particular the video subsystem, and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
776 open a window surface with a dimension of $640 x 480$ in 32-bit color
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
777 depths that has support for OpenGL rendering:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
778
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
779 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
780 > SDL_Init(SDL_INIT_VIDEO)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
781 > surface <- SDL_SetVideoMode(640,480,32,SDL_OPENGL)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
782 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
783
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
784 Next, we implement the application loop which updates the display repeatedly
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
785 and processes the event queue until a \emph{quit} request is
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
786 issued by the user via the window close button.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
787
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
788 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
789 > mainloop <- function()
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
790 {
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
791 ev <- new.struct(SDL_Event)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
792 quit <- FALSE
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
793 while(!quit) {
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
794 draw()
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
795 while(SDL_PollEvent(ev)) {
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
796 if (ev$type == SDL_QUIT) {
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
797 quit <- TRUE
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
798 }
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
799 }
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
800 }
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
801 }
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
802 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
803
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
804 SDL event processing is implemented by collecting events that occur in a
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
805 queue.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
806 Once per update frame, typical SDL applications poll the queue by
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
807 calling \code{SDL\_PollEvent} with a pointer to a user-allocated buffer
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
808 of C type \code{union SDL\_Event}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
809 Event records have a common type identifier which is set to \code{SDL\_QUIT}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
810 when a quit event has occurred e.g. when users press a close button on a window.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
811
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
812 Next, we implement our \code{draw} function making use of
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
813 the OpenGL 1.1 API. We clear the background with a blue color
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
814 and draw a light-green rectangle.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
815
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
816 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
817 > draw <- function()
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
818 {
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
819 glClearColor(0,0,1,0)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
820 glClear(GL_COLOR_BUFFER_BIT)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
821 glColor3f(0.5,1,0.5)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
822 glRectf(-0.5,-0.5,0.5,0.5)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
823 SDL_GL_SwapBuffers()
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
824 }
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
825 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
826
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
827 Now we can run the application mainloop.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
828
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
829 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
830 > mainloop()
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
831 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
832
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
833 To stop the application, we hit the close button of the window.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
834 A similar example is also available via \code{demo(SDL)}. Here the \code{draw} function
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
835 displays a rotating 3D cube depict in Figure \ref{fig:demo_SDL}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
836
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
837 \begin{figure}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
838 \centering
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
839 \includegraphics[scale=0.35]{img_SDL.png}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
840 \caption{\label{fig:demo_SDL}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
841 \code{demo(SDL)}}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
842 \end{figure}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
843
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
844 \code{demo(randomfield)} gives a slightly more scientific application of OpenGL and R:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
845 Random fields of 512x512 size are generated via blending of 5000 texture mapped 2D gaussian kernels.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
846 The \emph{frames per second} counter in the window title gives the number of matrices generated per second (see Figure \ref{fig:demo_randomfield}).
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
847 When clicking on the animation window, the current frame and matrix is passed to R and plotted.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
848 While several dozens of matrices are computed per second using OpenGL,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
849 it takes several seconds to plot a single matrix in R using \code{image()}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
850
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
851 \begin{figure}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
852 \centering
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
853 \includegraphics[scale=0.35]{img_randomfield.png}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
854 \caption{\label{fig:demo_randomfield}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
855 \code{demo(randomfield)}}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
856 \end{figure}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
857
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
858 \subsection{Parsing XML using Expat}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
859
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
860 In the second example, we use the Expat XML Parser library \citep{www:expat}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
861 \citep{Kim:2001:TSJ} to implement a stream-oriented XML parser suitable
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
862 for very large documents.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
863
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
864 The library, being very popular, is very likely to be
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
865 already installed on many OS distributions - otherwise it is
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
866 available from package repositories or can be built as a shared library
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
867 from source.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
868
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
869 In Expat, custom XML parsers are implemented by defining
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
870 functions that are registered as callbacks to be invoked on
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
871 events that occur during parsing, such as the start and end of XML tags.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
872 In our second example, we create a simple parser skeleton that
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
873 prints the start and end tag names.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
874
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
875 First we load R bindings for Expat via \code{dynport}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
876
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
877 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
878 > dynport(expat)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
879 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
880
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
881 Next we create an abstract parser object via the C function
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
882 \code{XML\_ParserCreate} that receives one argument of type C string
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
883 to specify a desired character encoding that overrides the document
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
884 encoding declaration. We want to pass a null pointer (\code{NULL}) here.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
885 In the \code{.dyncall} FFI C null pointer values for pointer types are
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
886 expressed via the R \code{NULL} value:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
887
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
888 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
889 > p <- XML_ParserCreate(NULL)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
890 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
891
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
892 The C interface for registration of start and end-tag event handler
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
893 callbacks is given below:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
894
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
895 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
896 /* Language C, from file expat.h: */
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
897 typedef void (*XML_StartElementHandler)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
898 (void *userData, const XML_Char *name,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
899 const XML_Char **atts);
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
900 typedef void (*XML_EndElementHandler)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
901 (void *userData, const XML_Char *name);
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
902 void XML_SetElementHandler(XML_Parser parser,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
903 XML_StartElementHandler start,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
904 XML_EndElementHandler end);
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
905 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
906
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
907 We implement the callbacks as R functions which print the event and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
908 tag name. They are wrapped as C callback pointers via \code{new.callback}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
909 using a matching \emph{callback signature}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
910 The second argument \code{name} of type C string in both callbacks, \code{XML\_StartElementHandler} and \code{XML\_EndElementHandler},
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
911 is of primnary interest ; this argument passes over the XML tag name.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
912 C strings are handled in a special way by the \code{.dyncall} FFI, because they
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
913 have to be copied as R \code{character} objects.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
914 The special type signature \code{'Z'} is used to denote a
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
915 C string type.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
916 The other arguments are simply denoted as untyped pointers using \code{'p'}:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
917
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
918 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
919 > start <- new.callback("pZp)v",
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
920 function(ignored1,tag,ignored2)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
921 cat("Start tag:", tag, "\n")
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
922 )
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
923 > end <- new.callback("pZ)v",
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
924 function(ignored,tag)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
925 cat("Stop tag:", tag, "\n")
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
926 )
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
927 > XML_SetElementHandler(p, start, end)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
928 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
929
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
930 To test the parser, we create a sample document stored in a \code{character}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
931 object named \code{text} and pass it to the parse function \code{XML\_Parse}:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
932
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
933 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
934 > text <- "<hello> <world> </world> </hello>"
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
935 > XML_Parse( p, text, nchar(text), 1)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
936 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
937
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
938 The resulting output is given below:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
939
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
940 \begin{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
941 Start tag: hello
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
942 Start tag: world
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
943 End tag: world
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
944 End tag: hello
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
945 \end{smallverbatim}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
946
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
947 Expat supports processing of very large XML documents in a chunk-based manner by
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
948 calling \code{XML\_Parse} several times, where the last argument is used
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
949 as indicator for the final chunk of the document.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
950
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
951 \section{Architecture}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
952
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
953 The core implementation of the FFI, callbacks and loading of
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
954 code are mainly based on the suite of libraries of the \emph{DynCall}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
955 project \citep{dyncall}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
956
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
957 \subsection{Dynamic calls}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
958
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
959 The FFI offered by \pkg{rdyncall} is based on the \pkg{dyncall}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
960 library, which provides an abstraction for making arbitrary
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
961 machine-level calls with support for multiple calling conventions
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
962 and most C argument- and return-types. \footnote{\emph{Inline} structure types are currently not fully supported.}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
963
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
964 For each processor architecture, the supported calling conventions
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
965 are abstracted in a \emph{Call Virtual Machine} (CallVM)
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
966 object. The \pkg{dyncall} library offers a universal C interface that can
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
967 be used from within scripting language interpreter contexts to build
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
968 up a machine-level call in a structured manner.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
969
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
970 A CallVM comprises a state machine and a call kernel. The state machine
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
971 is implemented in C and keeps track of internal buffers for pre-loading argument
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
972 values that get arranged for specific storage locations, such as stack or
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
973 special register sets according to the processor architecture and the chosen
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
974 calling conventions.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
975 The actual invocation of a foreign function call is conducted by
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
976 the Call Kernel - a small piece of code that is implemented in
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
977 Assembly and that provides a generic call facility for a particular
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
978 calling convention.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
979 It prepares machine-level calls by copying data to registers and to the
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
980 call stack according to the relevant calling convention, and finally
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
981 executes the machine call to a target address.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
982
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
983 From a scripting language interpreter perspective, the invocation of a
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
984 foreign function call through the CallVM is conducted in three consecutive
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
985 phases using the \pkg{dyncall} C API:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
986
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
987 \begin{enumerate}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
988 \item \emph{Setup Phase:} The desired calling convention has to be
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
989 chosen which, in most cases, is just the \emph{default C} calling convention.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
990 However, more specialized and platform-specific calling conventions are
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
991 available as well, in particular for the 32-Bit Windows OS.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
992 \item \emph{Argument Loading Phase:} Arguments are passed in a
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
993 \emph{left-to-right} order according to the declaration of the C/C++
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
994 function/method type declaration. Argument values are stored in buffers
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
995 according to the processor architecture and selected calling convention.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
996 \item \emph{Call and Return-Value Receive Phase:}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
997 A return-type specific call function is chosen and the target address
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
998 of the foreign code is passed, which gets called via the Call Kernel.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
999 \end{enumerate}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1000
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1001 The architecture makes it straight-forward to implement a FFI
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1002 for a dynamic language interpreter using a text parser for call signatures
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1003 to drive the conversion of arguments and results.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1004 Similar FFIs with a text-based interface have been implemented for other language
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1005 interpreters such as Ruby, Python and Lua. See the DynCall source repository \citep{dyncall}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1006
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1007 Both the C interface of dyncall and the signature format use the abstract
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1008 C/C++ type system and give no indication about the effective size of
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1009 a particular type. In experiments with several C APIs bound via \pkg{rdyncall}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1010 it turns out that the signatures do work cross-platform,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1011 if the fundamental type definitions of the C API do not change across platforms.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1012 In our tests and the presented examples, a wide range of
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1013 C APIs have this property and type signatures are valid across
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1014 platforms even when switching between 32- and 64-bit platforms.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1015
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1016 \subsection{Dynamic callbacks}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1017
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1018 The \pkg{dyncallback} library provides a framework to implement
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1019 dynamic callbacks for language interpreters to wrap scripting functions
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1020 as C function pointers.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1021 The framework offers a universal C interface for callback handler that
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1022 is implemented once for a particular interpreter.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1023 The handler receives callback calls from C and forwards the call,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1024 including conversion of arguments, to a scripting function.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1025
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1026 Handlers need to access machine-level arguments whose location
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1027 can be on the stack, or in registers,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1028 depending on the processor architecture and calling convention.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1029 For that reason, the handler interface receives an abstract argument
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1030 iterator that gives structured access to the arguments for
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1031 passing over to the high-level language.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1032 Call-backs are created via an interface that pools a handler,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1033 language context, scripting function reference,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1034 callback type-information and other user data into a
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1035 \emph{single} native C function pointer, such that even very
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1036 low-level C callbacks without user-supplied user-data can be
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1037 addressed with the underlying technique. \footnote{This includes
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1038 callbacks for sort routines of the Standard C library which lack user-data.}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1039
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1040 \subsection{Portability and Stability}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1041
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1042 The requirements for porting the \emph{DynCall} libraries to
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1043 a new processor and/or platform are high: The calling conventions of a target processor platform have to be studied in detail,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1044 state machines have to be implemented in C and a small amount of code has to be written in
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1045 Assembly which can be even non-portable across build tools on the same platform.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1046 Nevertheless \pkg{dyncall} (as of version 0.7) has support for many processor architectures such as
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1047 Intel i386 (x86), AMD 64 (x64), PowerPC 32-bit,ARM (including Thumb extension), MIPS 32/64-bit and SPARC 32/64-bit
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1048 including support for several platform-, processor- and compiler-specific calling conventions.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1049 \pkg{dyncallback} also supports major processor architectures such as Intel i386 (x86), AMD 64 (x64) and ARM and offers
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1050 partial support for PowerPC 32-bit (support for Mac OS X/Darwin).
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1051 Besides the processor architecture, the libraries are also explicitly ported and tested on
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1052 various OS such as Linux, Mac OS X, Windows, the BSD family, Solaris, Haiku, Minix and Plan9.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1053 Support for embedded platforms such as Playstation Portable, Nintendo DS and iPhone OS is available as well.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1054
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1055 \emph{DynCall} contains a suite of testing tools for quality assurance. Included are test-case generators written in
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1056 Lua and Python. Extreme call and callback scenarios are tested here to ensure correct passing of arguments and results.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1057 Before a release, the libraries and tests are built for a large set of architectures on
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1058 \pkg{DynOS} \citep{dynos} - a batch-build system using full system emulators such as
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1059 \pkg{QEmu}\citep{qemu} and \pkg{GXEmul}\citep{gxemul} and various operating-system images
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1060 to test release candidates and create pre-built binary releases of the library.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1061
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1062 \subsection{Text-based Signature Interfaces}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1063
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1064 A common property of the service interface presented here is the use of
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1065 signature text formats. Signatures are used
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1066 as descriptors for types, such as foreign function calls, callbacks and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1067 aggregate data types.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1068 The reasons that lead to the use of signatures as a high-level user-interface
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1069 to interact with such services are given next:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1070
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1071 \begin{enumerate}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1072 \item Cross-language interface: Text format interfaces are available across
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1073 high-level languages. Examples for cross-language text-based
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1074 interfaces include regular expressions or \code{printf}-style formatted output
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1075 descriptions.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1076
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1077 \item Developer-friendly:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1078 The simplicity and compactness of the text-format enables developers
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1079 to bridge with foreign code in interactive and rapid development
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1080 sessions.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1081 C type signatures can be derived by hand with minimum effort:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1082 Fundamental types are encoded with a single character and the
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1083 upper-case encodes an \code{unsigned} type.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1084
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1085 \item Machine-neutral:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1086 In contrast to binary encoded type libraries, the data format is not affected
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1087 by the endian model of the underlying platform.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1088
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1089 \item Parser-friendly:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1090 The signature format can be used as driver code to perform foreign function
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1091 calls. Implementations of parsers match the sequential
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1092 design of \pkg{dyncall}'s CallVM and \pkg{dyncallback}'s argument iterator interface.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1093 \end{enumerate}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1094
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1095 \subsection{Creation of DynPort files}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1096
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1097 In this section we describe the tool-chain that creates the
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1098 universal bindings called \emph{DynPort}. The process described
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1099 here is applied once on a build machine, the generated output
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1100 is used later at run-time across platforms to drive the
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1101 dynamic linkage and binding procedure.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1102 \emph{DynPort} files can be created automatically from
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1103 C header files using a tool-chain as depicted in
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1104 Figure \ref{fig:gen_dynport}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1105
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1106 \begin{figure}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1107 \centering
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1108 \includegraphics[scale=0.45]{img_gen_dynport.pdf}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1109 \caption{\label{fig:gen_dynport}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1110 Tool-chain to create \emph{DynPort} files from C headers}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1111 \end{figure}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1112
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1113 The tool-chain comprises several freely available components that
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1114 are briefly described next:
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1115 \pkg{GCC-XML} \citep{gccxml} is a modified version of the GCC compiler
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1116 which translates C sources to XML document.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1117 \pkg{xsltproc}, distributed as part of the \pkg{libxslt} library
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1118 \citep{libxslt}, is a XSLT processor that transforms XML documents to
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1119 XML, text or binary formats according to style-sheets written in
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1120 the \emph{XSL Transformations} \citep{Clark:01:XTV} language.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1121
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1122 To extract library binding specifications, a main C source file is created that
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1123 consists of one or more \code{\#include} statements that
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1124 reference library and/or system header files to process.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1125 The header files should have been previously installed on
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1126 the build machine.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1127 In a preprocessing phase, the GNU C Macro Processor is used to process
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1128 all \code{\#include} statements using standard system search paths
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1129 to create a concatenated \emph{All-In-One} source file free of any
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1130 \code{\#include} statements.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1131 GCC-XML transforms C header declarations to XML.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1132 A XSL style-sheet implements the transformation of XML to
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1133 type signature formats using a XSLT processor.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1134 C Macro \code{\#define} statements are handled separately by a custom
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1135 C Preprocessor implemented in C++ using the boost wave library \citep{boostwave}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1136 An optional filter stage is used to include only elements with
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1137 a certain pattern such as a common prefix usually found in many
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1138 libraries e.g. '\code{SDL\_}'.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1139 In a last step, the various fragments are assembled into a single
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1140 text-file which represents the \emph{DynPort} file.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1141 The overall build process is managed by \emph{make} files and a repository of recipes
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1142 has been setup to extend support for additional
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1143 dynports and libraries in a structured and coordinated way.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1144
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1145
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1146 \section{Summary and Outlook}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1147
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1148 This paper introduces the \pkg{rdyncall} package (Version 0.7.3 on CRAN as of this writing) that contributes an improved Foreign Function Interface for R.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1149 The FFI facilitates \emph{direct} invocation of foreign functions \emph{without} the need to compile additional wrapper in C.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1150 Based on the FFI, a dynamic cross-platform linkage framework to wrap and access \emph{whole} C interfaces of native libraries from R
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1151 is discussed.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1152 Instead of \emph{compiling} bindings for every library-and-language combination,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1153 R bindings of a library are created dynamically at run-time in a data-driven manner via
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1154 \emph{DynPort} files - a cross-platform universal type information format.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1155 C libraries are made accessible in R as though they were extension packages and
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1156 the R interface looks very similar to that of C.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1157 This enables system-level programming in R and brings a new wave of possibilities for R developers
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1158 such as using OpenGL directly in R across platforms as described in the example.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1159 An initial repository of \emph{DynPort}s for standard cross-platform portable
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1160 C libraries comes with the package.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1161
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1162 The implementation is based on libraries from the \emph{DynCall} project that implement non-trivial
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1163 facilities such as an abstraction to machine-level function calls supporting
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1164 multiple calling conventions and the handling of C callbacks from within scripting language interpreter environments.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1165 The libraries have been ported across major R platforms.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1166 Work is in progress to support missing architectures in \pkg{dyncallback} such as PowerPC System V 32-bit, PowerPC 64-bit, and, 32/64-bit MIPS and SPARC architectures.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1167 The handling of foreign aggregate data types, which is currently implemented in R and C,
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1168 is planned to be reimplemented in portable C as part of \emph{DynCall}, in cooperation with the developers of \emph{BridJ}\citep{bridj}.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1169 Currently, \emph{DynPort} files are written as R scripts with
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1170 inline text chunks created from the \emph{DynPort} tool chain.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1171 For the Lua Programming Language \citep{SPE::IerusalimschyFF1996}, a similar framework named \pkg{luadyncall} is in
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1172 development using a language-neutral format for \emph{DynPort} files.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1173 The need to install additional shared libraries still represents a hurdle for ordinary R users.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1174 We plan to find a common abstraction layer for installation systems, package managers and software distribution services
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1175 across OS-distributions, and to integrate meta installation information into the \emph{DynPort} file format.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1176
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1177 The \emph{DynPort} facility in \pkg{rdyncall} consitutes an initial step in building up an infrastructure between
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1178 scripting languages and C libraries.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1179 Analogous to the way in which R users enjoy quick access to the large pool of R software
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1180 managed by CRAN, we envision an archive network in which C library developers can distribute
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1181 their work across languages, and users get quick access to the pool of C libraries from within
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1182 scripting languages via automatic installation of precompiled components and using
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1183 universal type information for cross-platform and cross-language dynamic bindings.
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1184
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1185 \bibliography{FLI}
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1186
0cfcc391201f initial from svn dyncall-1745
Daniel Adler
parents:
diff changeset
1187 \end{document}