Wrapping C opaque pointers in Fortran 2003 in type-strict way

Opaque pointers are a programming technique used to hide implememtation details. In C, you can declare that a struct exists but don't tell the compiler anything else about it:

struct target;
struct target * new_target(void);
void shoot(struct target *);

At the cost of one more pointer access per function call you now have totally abstracted the inner details of the structure. You are now free to change them at will. The code that doesn't know the definition of the structure can only have pointers to objects of this incomplete type and distinguish between pointers of different types, but cannot peek inside:

struct foot;
struct foot * new_foot(void);

struct target * tgt = new_foot();
// warning: initialization from incompatible pointer type [-Wincompatible-pointer-types]
// error: dereferencing pointer to incomplete type ‘struct target’

Of course, reading the memory is not actually forbidden, as is type casting:

struct foot * f = new_foot();
char buf[256];
memcpy(buf, foot, 42); // the user still has to guess the size of the structure, though
// look ma, no warnings!
shoot((struct target*)foot);

If you find yourself questioning the use of such opaque pointers, have a look at FILE*-related functions in the standard C library. Depending on where you got it from, the actual FILE may store just a file descriptor and a buffer, or a dynamically allocated buffer that is pretending it's a file, or something completely different. And even if you only limit yourself to actual filesystems, by using FILE* you stop caring whether the underlying file descriptor is a HANDLE or an int and how exactly should you do I/O on it.

So there is no surprise that NLopt uses opaque pointers to hide the way it allocates and stores data:

struct nlopt_opt_s;
typedef struct nlopt_opt_s *nlopt_opt;

What if we want to call the library from Fortran? The documentation states:

The nlopt_opt type corresponds to integer*8. (Technically, we could use any type that is big enough to hold a pointer on all platforms; integer*8 is big enough for pointers on both 32-bit and 64-bit machines.)

NLopt supports very old programming language standards: if you omit a few algorithms from the library, you can build it with an ANSI C compiler and then call it from a FORTRAN 77 program. This is actually good (don't you hate it when a new app requires you to upgrade your libraries, your operating system, your computer, your spouse, and your cat?), but let's try to employ some techniques for writing cleaner and safer code brought to us by Fortran 90 (derived types) and Fortran 2003 (C interoperability).

The way that's usually recommended on the internet to wrap opaque pointers is just to declare them on the Fortran side as type(c_ptr), a type with all its fields private that corresponds to void *. Which is not wrong: according to the C standard, any pointer may be safely cast to void * and back and still compare equal to the original.

A concrete example:

nlopt_opt nlopt_create(nlopt_algorithm algorithm, unsigned n);

transforms to:

function nlopt_create(algo, n) result(opt) bind(c)
 use, intrinsic :: iso_c_binding
 type(c_ptr) :: opt
 ! assume that NLOPT_* algorithm enums were defined as enum, bind(c)
 integer(kind(NLOPT_NUM_ALGORITHMS)), value :: algo
 ! also, there are no unsigned integers in Fortran, but the size matches
 integer(kind=c_int), value :: n
end function

And this is, mostly, good enough: Fortran is usually written by real programmers who don't confuse one void* with another type(c_ptr) and if you actually have more than one kind of opaque C pointer in your program, why are you still writing Fortran? But let's try to make nlopt_opt a distinct type in Fortran anyway.

I couldn't express "this is a pointer to a user-defined type that's never actually defined" in Fortran in a way that would still be interoperable with C. We will have to create a fully-defined derived type and make it behave exactly as an opaque pointer would. Before we do that, let's prove a lemma:

typedef struct nlopt_opt_s *nlopt_opt;

// is the same thing as

typedef struct {
	void * nlopt_opt_s;
} nlopt_opt;

// for bit pattern purposes

C standard, § says:

A pointer to a structure object, suitably converted, points to its initial member (or if that member is a bit-field, then to the unit in which it resides), and vice versa.
There may be unnamed padding within a structure object, but not at its beginning.

So the beginning of our struct is guaranteed to be the same as beginning of the pointer we want to represent. What about the end? C standard, § is less optimistic:

There may be unnamed padding at the end of a structure or union

However, on the architectures I have access to (Linux and Windows on 32-bit and 64-bit Intel; Linux on 32-bit ARM) no padding is placed after the pointer because void* is self-aligned - alignment requirement of the pointer type is the same as the pointer size.

So a struct containing a single pointer has, indeed, the same size and bit pattern as just a pointer. (Unless you are using a weird architecture that does things differently. In that case, sorry.) The difference is that we can express such a struct as an interoperable derived Fortran type:

type, bind(c) :: nlopt_opt
 type(c_ptr) :: ptr
end type

Passing the structure by value is equivalent to passing the original pointer, um, by value. By marking the pointer private we ensure the original intent of opaque pointers: the implementation informaion is hidden from the library user. With that done, we can write a type-strict definition for all methods of the NLopt object:

!NLOPT_EXTERN(nlopt_opt) nlopt_create(nlopt_algorithm algorithm, unsigned n);
function nlopt_create(algo, n) bind(c) result(ret)
 import nlopt_opt, c_int
 type(nlopt_opt) :: ret
 integer(kind(NLOPT_NUM_ALGORITHMS)), value :: algo
 integer(c_int), value :: n
end function

!NLOPT_EXTERN(void) nlopt_destroy(nlopt_opt opt);
subroutine nlopt_destroy(opt) bind(c)
 import nlopt_opt
 type(nlopt_opt), value :: opt
end subroutine

Unfortunately, the code described above, despite working on my machine, is incorrect. While on some machines a struct whatever * gets passed in the same way a struct { void * ptr; } would, it is in no way guaranteed that rules for returning primitive and aggregate types would remain same, despite them having the same size and alignment.

Therefore, the correct solution is either to continue using type(c_ptr) with no distinction between kinds of void*, or to create a proper devived type and write a bunch of wrapper functions, hoping that they would get inlined by other code. You can also declare a finalizer or even rewrite all C procedure calls into object methods:

type :: nlopt_opt
 type(c_ptr) :: ptr = C_NULL_PTR
 procedure :: set_xtol_rel ! add more methods here
 final :: destroy
end type nlopt_opt

subroutine destroy(this)
 class(nlopt_opt) :: this
  subroutine nlopt_destroy(opt) bind(c)
   import nlopt_opt
   type(nlopt_opt), value :: opt
  end subroutine
 end interface

 call nlopt_destroy(this%ptr)
end subroutine destroy

Now package it all in a Fortran 90 module and you are done.

Unless otherwise specified, contents of this blog are covered by CC BY-NC-SA 4.0 license (or a later version).