This is Edition 0.10, last updated 2001-07-06, of The GNU C Library Reference Manual, for Version 2.2.x of the GNU C Library.
Appendices
Indices
--- The Detailed Node Listing ---
Introduction
Standards and Portability
Using the Library
Error Reporting
Memory
brk
, sbrk
Memory Allocation
malloc
facility allows fully general
dynamic allocation.
Unconstrained Allocation
malloc
.
malloc
. xmalloc
.
free
to free a block you
got with malloc
.
realloc
to make a block
bigger or smaller.
calloc
to allocate a
block and clear it.
mallopt
to adjust allocation
parameters.
malloc
.
malloc
and related functions.
Allocation Debugging
Obstacks
Variable Size Automatic
alloca
.
alloca
.
alloca
.
Locking Pages
Character Handling
String and Array Utilities
Argz and Envz Vectors
Character Set Handling
Restartable multibyte conversion
Non-reentrant Conversion
Generic Charset Conversion
iconv
example.
iconv
Implementations.
iconv
Implementation in the GNU C
library.
Locales
Locale Information
localeconv
.
nl_langinfo
.
The Lame Way to Locale Data
$
).
Message Translation
catgets
family of functions.
gettext
family of functions.
Message catalogs a la X/Open
catgets
function family.
catgets
interface.
The Uniforum approach
gettext
family of functions.
gettext
.
Message catalogs with gettext
gettext
uses.
gettext
in GUI programs.
gettext
works.
Searching and Sorting
bsearch
function.
qsort
function.
hsearch
function.
tsearch
function.
Pattern Matching
Globbing
glob
.
glob
.
glob
.
Regular Expressions
regcomp
to prepare to match.
regcomp
.
regexec
to match the compiled
pattern that you get from regcomp
.
Word Expansion
wordexp
.
wordexp
.
I/O Overview
I/O Concepts
File Names
I/O on Streams
printf
and related functions.
printf
and friends.
scanf
and related functions.
Unreading
ungetc
to do unreading.
Formatted Output
vprintf
and friends.
parse_printf_format
.
Customizing Printf
register_printf_function
to register a new output conversion.
register_printf_function
.
printf
handler function.
printf
handlers.
Formatted Input
malloc
the buffer.
vscanf
and friends.
Stream Buffering
Other Kinds of Streams
Custom Streams
Formatted Messages
fmtmsg
function.
fmtmsg
and addseverity
.
Low-Level I/O
Stream/Descriptor Precautions
Asynchronous I/O
File Status Flags
open
.
File System Interface
Accessing Directories
File Attributes
Pipes and FIFOs
pipe
function.
Sockets
Socket Addresses
struct sockaddr
.
Local Namespace
Internet Namespace
Host Addresses
Open/Close Sockets
Connections
Transferring Data
send
.
recv
.
send
and recv
.
Datagrams
Inetd
Socket Options
Low-Level Terminal Interface
Terminal Modes
struct termios
and
related types.
Special Characters
Pseudo-Terminals
Syslog
Submitting Syslog Messages
Mathematics
Pseudo-Random Numbers
rand
and friends.
random
and friends.
drand48
and friends.
Arithmetic
Floating Point Errors
Arithmetic Functions
Parsing of Numbers
Date and Time
Processor And CPU Time
clock
function.
times
function.
Calendar Time
Parsing Date and Time
Resource Usage And Limitation
Priority
Traditional Scheduling
Memory Resources
Non-Local Exits
Signal Handling
open
,
read
, write
and other functions.
Concepts of Signals
Standard Signals
Signal Actions
signal
function.
sigaction
function.
Defining Handlers
Atomic Data Access
Generating Signals
kill
.
kill
for Communication.
Blocking Signals
Waiting for a Signal
pause
.
BSD Signal Handling
Program Basics
Program Arguments
Parsing Program Arguments
getopt
.
argp_parse
.
mount
.
Environment Variables
Program Termination
exit
, a
process terminates normally.
exit status
provides information
about why the process terminated.
abort
function causes
abnormal program termination.
Processes
Job Control
Implementing a Shell
Functions for Job Control
Name Service Switch
NSS Configuration File
NSS Module Internals
Extending NSS
Users and Groups
User Accounting Database
User Database
Group Database
Netgroup Database
System Management
Filesystem Handling
Mount Information
fstab
file
mtab
file
System Configuration
Sysconf
sysconf
.
sysconf
can read.
sysconf
and the parameter
macros properly together.
Cryptographic Functions
Debugging Support
Language Features
assert
to abort if
something ``impossible'' happens.
NULL
.
Variadic Functions
How Variadic
Data Type Measurements
Floating Type Macros
Installation
Maintenance
Porting
sysdeps
hierarchy.
The C language provides no built-in facilities for performing such common operations as input/output, memory management, string manipulation, and the like. Instead, these facilities are defined in a standard library, which you compile and link with your programs.
The GNU C library, described in this document, defines all of the library functions that are specified by the ISO C standard, as well as additional features specific to POSIX and other derivatives of the Unix operating system, and extensions specific to the GNU system.
The purpose of this manual is to tell you how to use the facilities of the GNU library. We have mentioned which features belong to which standards to help you identify things that are potentially non-portable to other systems. But the emphasis in this manual is not on strict portability.
This manual is written with the assumption that you are at least somewhat familiar with the C programming language and basic programming concepts. Specifically, familiarity with ISO standard C (see ISO C), rather than "traditional" pre-ISO C dialects, is assumed.
The GNU C library includes several header files, each of which
provides definitions and declarations for a group of related facilities;
this information is used by the C compiler when processing your program.
For example, the header file stdio.h
declares facilities for
performing input and output, and the header file string.h
declares string processing utilities. The organization of this manual
generally follows the same division as the header files.
If you are reading this manual for the first time, you should read all of the introductory material and skim the remaining chapters. There are a lot of functions in the GNU C library and it's not realistic to expect that you will be able to remember exactly how to use each and every one of them. It's more important to become generally familiar with the kinds of facilities that the library provides, so that when you are writing your programs you can recognize when to make use of library functions, and where in this manual you can find more specific information about them.
This section discusses the various standards and other sources that the GNU C library is based upon. These sources include the ISO C and POSIX standards, and the System V and Berkeley Unix implementations.
The primary focus of this manual is to tell you how to make effective use of the GNU library facilities. But if you are concerned about making your programs compatible with these standards, or portable to operating systems other than GNU, this can affect how you use the library. This section gives you an overview of these standards, so that you will know what they are when they are mentioned in other parts of the manual.
See Library Summary, for an alphabetical list of the functions and other symbols provided by the library. This list also states which standards each function or symbol comes from.
The GNU C library is compatible with the C standard adopted by the American National Standards Institute (ANSI): American National Standard X3.159-1989--"ANSI C" and later by the International Standardization Organization (ISO): ISO/IEC 9899:1990, "Programming languages--C". We here refer to the standard as ISO C since this is the more general standard in respect of ratification. The header files and library facilities that make up the GNU library are a superset of those specified by the ISO C standard.
If you are concerned about strict adherence to the ISO C standard, you
should use the -ansi
option when you compile your programs with
the GNU C compiler. This tells the compiler to define only ISO
standard features from the library header files, unless you explicitly
ask for additional features. See Feature Test Macros, for
information on how to do this.
Being able to restrict the library to include only ISO C features is important because ISO C puts limitations on what names can be defined by the library implementation, and the GNU extensions don't fit these limitations. See Reserved Names, for more information about these restrictions.
This manual does not attempt to give you complete details on the differences between ISO C and older dialects. It gives advice on how to write programs to work portably under multiple C dialects, but does not aim for completeness.
The GNU library is also compatible with the ISO POSIX family of standards, known more formally as the Portable Operating System Interface for Computer Environments (ISO/IEC 9945). They were also published as ANSI/IEEE Std 1003. POSIX is derived mostly from various versions of the Unix operating system.
The library facilities specified by the POSIX standards are a superset of those required by ISO C; POSIX specifies additional features for ISO C functions, as well as specifying new additional functions. In general, the additional requirements and functionality defined by the POSIX standards are aimed at providing lower-level support for a particular kind of operating system environment, rather than general programming language support which can run in many diverse operating system environments.
The GNU C library implements all of the functions specified in ISO/IEC 9945-1:1996, the POSIX System Application Program Interface, commonly referred to as POSIX.1. The primary extensions to the ISO C facilities specified by this standard include file system interface primitives (see File System Interface), device-specific terminal control functions (see Low-Level Terminal Interface), and process control functions (see Processes).
Some facilities from ISO/IEC 9945-2:1993, the POSIX Shell and Utilities standard (POSIX.2) are also implemented in the GNU library. These include utilities for dealing with regular expressions and other pattern matching facilities (see Pattern Matching).
The GNU C library defines facilities from some versions of Unix which are not formally standardized, specifically from the 4.2 BSD, 4.3 BSD, and 4.4 BSD Unix systems (also known as Berkeley Unix) and from SunOS (a popular 4.2 BSD derivative that includes some Unix System V functionality). These systems support most of the ISO C and POSIX facilities, and 4.4 BSD and newer releases of SunOS in fact support them all.
The BSD facilities include symbolic links (see Symbolic Links), the
select
function (see Waiting for I/O), the BSD signal
functions (see BSD Signal Handling), and sockets (see Sockets).
The System V Interface Description (SVID) is a document describing the AT&T Unix System V operating system. It is to some extent a superset of the POSIX standard (see POSIX).
The GNU C library defines most of the facilities required by the SVID that are not also required by the ISO C or POSIX standards, for compatibility with System V Unix and other Unix systems (such as SunOS) which include these facilities. However, many of the more obscure and less generally useful facilities required by the SVID are not included. (In fact, Unix System V itself does not provide them all.)
The supported facilities from System V include the methods for
inter-process communication and shared memory, the hsearch
and
drand48
families of functions, fmtmsg
and several of the
mathematical functions.
The X/Open Portability Guide, published by the X/Open Company, Ltd., is a more general standard than POSIX. X/Open owns the Unix copyright and the XPG specifies the requirements for systems which are intended to be a Unix system.
The GNU C library complies to the X/Open Portability Guide, Issue 4.2, with all extensions common to XSI (X/Open System Interface) compliant systems and also all X/Open UNIX extensions.
The additions on top of POSIX are mainly derived from functionality available in System V and BSD systems. Some of the really bad mistakes in System V systems were corrected, though. Since fulfilling the XPG standard with the Unix extensions is a precondition for getting the Unix brand chances are good that the functionality is available on commercial systems.
This section describes some of the practical issues involved in using the GNU C library.
Libraries for use by C programs really consist of two parts: header files that define types and macros and declare variables and functions; and the actual library or archive that contains the definitions of the variables and functions.
(Recall that in C, a declaration merely provides information that a function or variable exists and gives its type. For a function declaration, information about the types of its arguments might be provided as well. The purpose of declarations is to allow the compiler to correctly process references to the declared variables and functions. A definition, on the other hand, actually allocates storage for a variable or says what a function does.)
In order to use the facilities in the GNU C library, you should be sure that your program source files include the appropriate header files. This is so that the compiler has declarations of these facilities available and can correctly process references to them. Once your program has been compiled, the linker resolves these references to the actual definitions provided in the archive file.
Header files are included into a program source file by the
#include
preprocessor directive. The C language supports two
forms of this directive; the first,
#include "header"
is typically used to include a header file header that you write
yourself; this would contain definitions and declarations describing the
interfaces between the different parts of your particular application.
By contrast,
#include <file.h>
is typically used to include a header file file.h
that contains
definitions and declarations for a standard library. This file would
normally be installed in a standard place by your system administrator.
You should use this second form for the C library header files.
Typically, #include
directives are placed at the top of the C
source file, before any other code. If you begin your source files with
some comments explaining what the code in the file does (a good idea),
put the #include
directives immediately afterwards, following the
feature test macro definition (see Feature Test Macros).
For more information about the use of header files and #include
directives, see Header Files.
The GNU C library provides several header files, each of which contains the type and macro definitions and variable and function declarations for a group of related facilities. This means that your programs may need to include several header files, depending on exactly which facilities you are using.
Some library header files include other library header files automatically. However, as a matter of programming style, you should not rely on this; it is better to explicitly include all the header files required for the library facilities you are using. The GNU C library header files have been written in such a way that it doesn't matter if a header file is accidentally included more than once; including a header file a second time has no effect. Likewise, if your program needs to include multiple header files, the order in which they are included doesn't matter.
Compatibility Note: Inclusion of standard header files in any order and any number of times works in any ISO C implementation. However, this has traditionally not been the case in many older C implementations.
Strictly speaking, you don't have to include a header file to use a function it declares; you could declare the function explicitly yourself, according to the specifications in this manual. But it is usually better to include the header file because it may define types and macros that are not otherwise available and because it may define more efficient macro replacements for some functions. It is also a sure way to have the correct declaration.
If we describe something as a function in this manual, it may have a macro definition as well. This normally has no effect on how your program runs--the macro definition does the same thing as the function would. In particular, macro equivalents for library functions evaluate arguments exactly once, in the same way that a function call would. The main reason for these macro definitions is that sometimes they can produce an inline expansion that is considerably faster than an actual function call.
Taking the address of a library function works even if it is also defined as a macro. This is because, in this context, the name of the function isn't followed by the left parenthesis that is syntactically necessary to recognize a macro call.
You might occasionally want to avoid using the macro definition of a function--perhaps to make your program easier to debug. There are two ways you can do this:
#undef
preprocessor directive, unless otherwise stated
explicitly in the description of that facility.
For example, suppose the header file stdlib.h
declares a function
named abs
with
extern int abs (int);
and also provides a macro definition for abs
. Then, in:
#include <stdlib.h> int f (int *i) { return abs (++*i); }
the reference to abs
might refer to either a macro or a function.
On the other hand, in each of the following examples the reference is
to a function and not a macro.
#include <stdlib.h> int g (int *i) { return (abs) (++*i); } #undef abs int h (int *i) { return abs (++*i); }
Since macro definitions that double for a function behave in exactly the same way as the actual function version, there is usually no need for any of these methods. In fact, removing macro definitions usually just makes your program slower.
The names of all library types, macros, variables and functions that come from the ISO C standard are reserved unconditionally; your program may not redefine these names. All other library names are reserved if your program explicitly includes the header file that defines or declares them. There are several reasons for these restrictions:
exit
to do something completely different from
what the standard exit
function does, for example. Preventing
this situation helps to make your programs easier to understand and
contributes to modularity and maintainability.
In addition to the names documented in this manual, reserved names
include all external identifiers (global functions and variables) that
begin with an underscore (_
) and all identifiers regardless of
use that begin with either two underscores or an underscore followed by
a capital letter are reserved names. This is so that the library and
header files can define functions, variables, and macros for internal
purposes without risk of conflict with names in user programs.
Some additional classes of identifier names are reserved for future extensions to the C language or the POSIX.1 environment. While using these names for your own purposes right now might not cause a problem, they do raise the possibility of conflict with future versions of the C or POSIX standards, so you should avoid these names.
E
followed a digit or uppercase
letter may be used for additional error code names. See Error Reporting.
is
or to
followed by a
lowercase letter may be used for additional character testing and
conversion functions. See Character Handling.
LC_
followed by an uppercase letter may be
used for additional macros specifying locale attributes.
See Locales.
f
or l
are reserved for corresponding
functions that operate on float
and long double
arguments,
respectively.
SIG
followed by an uppercase letter are
reserved for additional signal names. See Standard Signals.
SIG_
followed by an uppercase letter are
reserved for additional signal actions. See Basic Signal Handling.
str
, mem
, or wcs
followed by a
lowercase letter are reserved for additional string and array functions.
See String and Array Utilities.
_t
are reserved for additional type names.
In addition, some individual header files reserve names beyond those that they actually define. You only need to worry about these restrictions if your program includes that particular header file.
dirent.h
reserves names prefixed with
d_
.
fcntl.h
reserves names prefixed with
l_
, F_
, O_
, and S_
.
grp.h
reserves names prefixed with gr_
.
limits.h
reserves names suffixed with _MAX
.
pwd.h
reserves names prefixed with pw_
.
signal.h
reserves names prefixed with sa_
and SA_
.
sys/stat.h
reserves names prefixed with st_
and S_
.
sys/times.h
reserves names prefixed with tms_
.
termios.h
reserves names prefixed with c_
,
V
, I
, O
, and TC
; and names prefixed with
B
followed by a digit.
The exact set of features available when you compile a source file is controlled by which feature test macros you define.
If you compile your programs using gcc -ansi
, you get only the
ISO C library features, unless you explicitly request additional
features by defining one or more of the feature macros.
See GNU CC Command Options,
for more information about GCC options.
You should define these macros by using #define
preprocessor
directives at the top of your source code files. These directives
must come before any #include
of a system header file. It
is best to make them the very first thing in the file, preceded only by
comments. You could also use the -D
option to GCC, but it's
better if you make the source files indicate their own meaning in a
self-contained way.
This system exists to allow the library to conform to multiple standards.
Although the different standards are often described as supersets of each
other, they are usually incompatible because larger standards require
functions with names that smaller ones reserve to the user program. This
is not mere pedantry -- it has been a problem in practice. For instance,
some non-GNU programs define functions named getline
that have
nothing to do with this library's getline
. They would not be
compilable if all features were enabled indiscriminately.
This should not be used to verify that a program conforms to a limited standard. It is insufficient for this purpose, as it will not protect you from including header files outside the standard, or relying on semantics undefined within the standard.
_POSIX_SOURCE | Macro |
If you define this macro, then the functionality from the POSIX.1
standard (IEEE Standard 1003.1) is available, as well as all of the
ISO C facilities.
The state of |
_POSIX_C_SOURCE | Macro |
Define this macro to a positive integer to control which POSIX
functionality is made available. The greater the value of this macro,
the more functionality is made available.
If you define this macro to a value greater than or equal to If you define this macro to a value greater than or equal to If you define this macro to a value greater than or equal to Greater values for |
_BSD_SOURCE | Macro |
If you define this macro, functionality derived from 4.3 BSD Unix is
included as well as the ISO C, POSIX.1, and POSIX.2 material.
Some of the features derived from 4.3 BSD Unix conflict with the corresponding features specified by the POSIX.1 standard. If this macro is defined, the 4.3 BSD definitions take precedence over the POSIX definitions. Due to the nature of some of the conflicts between 4.3 BSD and POSIX.1,
you need to use a special BSD compatibility library when linking
programs compiled for BSD compatibility. This is because some functions
must be defined in two different ways, one of them in the normal C
library, and one of them in the compatibility library. If your program
defines |
_SVID_SOURCE | Macro |
If you define this macro, functionality derived from SVID is included as well as the ISO C, POSIX.1, POSIX.2, and X/Open material. |
_XOPEN_SOURCE | Macro |
_XOPEN_SOURCE_EXTENDED | Macro |
If you define this macro, functionality described in the X/Open
Portability Guide is included. This is a superset of the POSIX.1 and
POSIX.2 functionality and in fact _POSIX_SOURCE and
_POSIX_C_SOURCE are automatically defined.
As the unification of all Unices, functionality only available in BSD and SVID is also included. If the macro If the macro |
_LARGEFILE_SOURCE | Macro |
If this macro is defined some extra functions are available which
rectify a few shortcomings in all previous standards. Specifically,
the functions fseeko and ftello are available. Without
these functions the difference between the ISO C interface
(fseek , ftell ) and the low-level POSIX interface
(lseek ) would lead to problems.
This macro was introduced as part of the Large File Support extension (LFS). |
_LARGEFILE64_SOURCE | Macro |
If you define this macro an additional set of functions is made available
which enables 32 bit systems to use files of sizes beyond
the usual limit of 2GB. This interface is not available if the system
does not support files that large. On systems where the natural file
size limit is greater than 2GB (i.e., on 64 bit systems) the new
functions are identical to the replaced functions.
The new functionality is made available by a new set of types and
functions which replace the existing ones. The names of these new objects
contain This macro was introduced as part of the Large File Support extension
(LFS). It is a transition interface for the period when 64 bit
offsets are not generally used (see |
_FILE_OFFSET_BITS | Macro |
This macro determines which file system interface shall be used, one
replacing the other. Whereas _LARGEFILE64_SOURCE makes the 64 bit interface available as an additional interface,
_FILE_OFFSET_BITS allows the 64 bit interface to
replace the old interface.
If If the macro is defined to the value This macro should only be selected if the system provides mechanisms for
handling large files. On 64 bit systems this macro has no effect
since the This macro was introduced as part of the Large File Support extension (LFS). |
_ISOC99_SOURCE | Macro |
Until the revised ISO C standard is widely adopted the new features
are not automatically enabled. The GNU libc nevertheless has a complete
implementation of the new standard and to enable the new features the
macro _ISOC99_SOURCE should be defined.
|
_GNU_SOURCE | Macro |
If you define this macro, everything is included: ISO C89, ISO C99, POSIX.1, POSIX.2, BSD, SVID, X/Open, LFS, and GNU extensions. In
the cases where POSIX.1 conflicts with BSD, the POSIX definitions take
precedence.
If you want to get the full effect of #define _GNU_SOURCE #define _BSD_SOURCE #define _SVID_SOURCE Note that if you do this, you must link your program with the BSD
compatibility library by passing the |
_REENTRANT | Macro |
_THREAD_SAFE | Macro |
If you define one of these macros, reentrant versions of several functions get
declared. Some of the functions are specified in POSIX.1c but many others
are only available on a few other systems or are unique to GNU libc.
The problem is the delay in the standardization of the thread safe C library
interface.
Unlike on some other systems, no special version of the C library must be used for linking. There is only one version but while compiling this it must have been specified to compile as thread safe. |
We recommend you use _GNU_SOURCE
in new programs. If you don't
specify the -ansi
option to GCC and don't define any of these
macros explicitly, the effect is the same as defining
_POSIX_C_SOURCE
to 2 and _POSIX_SOURCE
,
_SVID_SOURCE
, and _BSD_SOURCE
to 1.
When you define a feature test macro to request a larger class of features,
it is harmless to define in addition a feature test macro for a subset of
those features. For example, if you define _POSIX_C_SOURCE
, then
defining _POSIX_SOURCE
as well has no effect. Likewise, if you
define _GNU_SOURCE
, then defining either _POSIX_SOURCE
or
_POSIX_C_SOURCE
or _SVID_SOURCE
as well has no effect.
Note, however, that the features of _BSD_SOURCE
are not a subset of
any of the other feature test macros supported. This is because it defines
BSD features that take precedence over the POSIX features that are
requested by the other macros. For this reason, defining
_BSD_SOURCE
in addition to the other feature test macros does have
an effect: it causes the BSD features to take priority over the conflicting
POSIX features.
Here is an overview of the contents of the remaining chapters of this manual.
sizeof
operator and the symbolic constant NULL
, how to write functions
accepting variable numbers of arguments, and constants describing the
ranges and other properties of the numerical types. There is also a simple
debugging mechanism which allows you to put assertions in your code, and
have diagnostic messages printed if the tests fail.
isspace
) and functions for
performing case conversion.
FILE *
objects). These are the normal C library functions
from stdio.h
.
char
data type.
setjmp
and
longjmp
functions. These functions provide a facility for
goto
-like jumps which can jump from one function to another.
If you already know the name of the facility you are interested in, you can look it up in Library Summary. This gives you a summary of its syntax and a pointer to where you can find a more detailed description. This appendix is particularly useful if you just want to verify the order and type of arguments to a function, for example. It also tells you what standard or system each function, variable, or macro is derived from.
Many functions in the GNU C library detect and report error conditions, and sometimes your programs need to check for these error conditions. For example, when you open an input file, you should verify that the file was actually opened correctly, and print an error message or take other appropriate action if the call to the library function failed.
This chapter describes how the error reporting facility works. Your
program should include the header file errno.h
to use this
facility.
Most library functions return a special value to indicate that they have
failed. The special value is typically -1
, a null pointer, or a
constant such as EOF
that is defined for that purpose. But this
return value tells you only that an error has occurred. To find out
what kind of error it was, you need to look at the error code stored in the
variable errno
. This variable is declared in the header file
errno.h
.
volatile int errno | Variable |
The variable errno contains the system error number. You can
change the value of errno .
Since The initial value of Many library functions can set Portability Note: ISO C specifies There are a few library functions, like |
All the error codes have symbolic names; they are macros defined in
errno.h
. The names start with E
and an upper-case
letter or digit; you should consider names of this form to be
reserved names. See Reserved Names.
The error code values are all positive integers and are all distinct,
with one exception: EWOULDBLOCK
and EAGAIN
are the same.
Since the values are distinct, you can use them as labels in a
switch
statement; just don't use both EWOULDBLOCK
and
EAGAIN
. Your program should not make any other assumptions about
the specific values of these symbolic constants.
The value of errno
doesn't necessarily have to correspond to any
of these macros, since some library functions might return other error
codes of their own for other situations. The only values that are
guaranteed to be meaningful for a particular library function are the
ones that this manual lists for that function.
On non-GNU systems, almost any system call can return EFAULT
if
it is given an invalid pointer as an argument. Since this could only
happen as a result of a bug in your program, and since it will not
happen on the GNU system, we have saved space by not mentioning
EFAULT
in the descriptions of individual functions.
In some Unix systems, many system calls can also return EFAULT
if
given as an argument a pointer into the stack, and the kernel for some
obscure reason fails in its attempt to extend the stack. If this ever
happens, you should probably try using statically or dynamically
allocated memory instead of stack memory on that system.
The error code macros are defined in the header file errno.h
.
All of them expand into integer constant values. Some of these error
codes can't occur on the GNU system, but they can occur using the GNU
library on other systems.
int EPERM | Macro |
Operation not permitted; only the owner of the file (or other resource) or processes with special privileges can perform the operation. |
int ENOENT | Macro |
No such file or directory. This is a "file doesn't exist" error for ordinary files that are referenced in contexts where they are expected to already exist. |
int ESRCH | Macro |
No process matches the specified process ID. |
int EINTR | Macro |
Interrupted function call; an asynchronous signal occurred and prevented
completion of the call. When this happens, you should try the call
again.
You can choose to have functions resume after a signal that is handled,
rather than failing with |
int EIO | Macro |
Input/output error; usually used for physical read or write errors. |
int ENXIO | Macro |
No such device or address. The system tried to use the device represented by a file you specified, and it couldn't find the device. This can mean that the device file was installed incorrectly, or that the physical device is missing or not correctly attached to the computer. |
int E2BIG | Macro |
Argument list too long; used when the arguments passed to a new program
being executed with one of the exec functions (see Executing a File) occupy too much memory space. This condition never arises in the
GNU system.
|
int ENOEXEC | Macro |
Invalid executable file format. This condition is detected by the
exec functions; see Executing a File.
|
int EBADF | Macro |
Bad file descriptor; for example, I/O on a descriptor that has been closed or reading from a descriptor open only for writing (or vice versa). |
int ECHILD | Macro |
There are no child processes. This error happens on operations that are supposed to manipulate child processes, when there aren't any processes to manipulate. |
int EDEADLK | Macro |
Deadlock avoided; allocating a system resource would have resulted in a deadlock situation. The system does not guarantee that it will notice all such situations. This error means you got lucky and the system noticed; it might just hang. See File Locks, for an example. |
int ENOMEM | Macro |
No memory available. The system cannot allocate more virtual memory because its capacity is full. |
int EACCES | Macro |
Permission denied; the file permissions do not allow the attempted operation. |
int EFAULT | Macro |
Bad address; an invalid pointer was detected. In the GNU system, this error never happens; you get a signal instead. |
int ENOTBLK | Macro |
A file that isn't a block special file was given in a situation that requires one. For example, trying to mount an ordinary file as a file system in Unix gives this error. |
int EBUSY | Macro |
Resource busy; a system resource that can't be shared is already in use. For example, if you try to delete a file that is the root of a currently mounted filesystem, you get this error. |
int EEXIST | Macro |
File exists; an existing file was specified in a context where it only makes sense to specify a new file. |
int EXDEV | Macro |
An attempt to make an improper link across file systems was detected.
This happens not only when you use link (see Hard Links) but
also when you rename a file with rename (see Renaming Files).
|
int ENODEV | Macro |
The wrong type of device was given to a function that expects a particular sort of device. |
int ENOTDIR | Macro |
A file that isn't a directory was specified when a directory is required. |
int EISDIR | Macro |
File is a directory; you cannot open a directory for writing, or create or remove hard links to it. |
int EINVAL | Macro |
Invalid argument. This is used to indicate various kinds of problems with passing the wrong argument to a library function. |
int EMFILE | Macro |
The current process has too many files open and can't open any more.
Duplicate descriptors do count toward this limit.
In BSD and GNU, the number of open files is controlled by a resource
limit that can usually be increased. If you get this error, you might
want to increase the |
int ENFILE | Macro |
There are too many distinct file openings in the entire system. Note that any number of linked channels count as just one file opening; see Linked Channels. This error never occurs in the GNU system. |
int ENOTTY | Macro |
Inappropriate I/O control operation, such as trying to set terminal modes on an ordinary file. |
int ETXTBSY | Macro |
An attempt to execute a file that is currently open for writing, or write to a file that is currently being executed. Often using a debugger to run a program is considered having it open for writing and will cause this error. (The name stands for "text file busy".) This is not an error in the GNU system; the text is copied as necessary. |
int EFBIG | Macro |
File too big; the size of a file would be larger than allowed by the system. |
int ENOSPC | Macro |
No space left on device; write operation on a file failed because the disk is full. |
int ESPIPE | Macro |
Invalid seek operation (such as on a pipe). |
int EROFS | Macro |
An attempt was made to modify something on a read-only file system. |
int EMLINK | Macro |
Too many links; the link count of a single file would become too large.
rename can cause this error if the file being renamed already has
as many links as it can take (see Renaming Files).
|
int EPIPE | Macro |
Broken pipe; there is no process reading from the other end of a pipe.
Every library function that returns this error code also generates a
SIGPIPE signal; this signal terminates the program if not handled
or blocked. Thus, your program will never actually see EPIPE
unless it has handled or blocked SIGPIPE .
|
int EDOM | Macro |
Domain error; used by mathematical functions when an argument value does not fall into the domain over which the function is defined. |
int ERANGE | Macro |
Range error; used by mathematical functions when the result value is not representable because of overflow or underflow. |
int EAGAIN | Macro |
Resource temporarily unavailable; the call might work if you try again
later. The macro EWOULDBLOCK is another name for EAGAIN ;
they are always the same in the GNU C library.
This error can happen in a few different situations:
|
int EWOULDBLOCK | Macro |
In the GNU C library, this is another name for EAGAIN (above).
The values are always the same, on every operating system.
C libraries in many older Unix systems have |
int EINPROGRESS | Macro |
An operation that cannot complete immediately was initiated on an object
that has non-blocking mode selected. Some functions that must always
block (such as connect ; see Connecting) never return
EAGAIN . Instead, they return EINPROGRESS to indicate that
the operation has begun and will take some time. Attempts to manipulate
the object before the call completes return EALREADY . You can
use the select function to find out when the pending operation
has completed; see Waiting for I/O.
|
int EALREADY | Macro |
An operation is already in progress on an object that has non-blocking mode selected. |
int ENOTSOCK | Macro |
A file that isn't a socket was specified when a socket is required. |
int EMSGSIZE | Macro |
The size of a message sent on a socket was larger than the supported maximum size. |
int EPROTOTYPE | Macro |
The socket type does not support the requested communications protocol. |
int ENOPROTOOPT | Macro |
You specified a socket option that doesn't make sense for the particular protocol being used by the socket. See Socket Options. |
int EPROTONOSUPPORT | Macro |
The socket domain does not support the requested communications protocol (perhaps because the requested protocol is completely invalid). See Creating a Socket. |
int ESOCKTNOSUPPORT | Macro |
The socket type is not supported. |
int EOPNOTSUPP | Macro |
The operation you requested is not supported. Some socket functions don't make sense for all types of sockets, and others may not be implemented for all communications protocols. In the GNU system, this error can happen for many calls when the object does not support the particular operation; it is a generic indication that the server knows nothing to do for that call. |
int EPFNOSUPPORT | Macro |
The socket communications protocol family you requested is not supported. |
int EAFNOSUPPORT | Macro |
The address family specified for a socket is not supported; it is inconsistent with the protocol being used on the socket. See Sockets. |
int EADDRINUSE | Macro |
The requested socket address is already in use. See Socket Addresses. |
int EADDRNOTAVAIL | Macro |
The requested socket address is not available; for example, you tried to give a socket a name that doesn't match the local host name. See Socket Addresses. |
int ENETDOWN | Macro |
A socket operation failed because the network was down. |
int ENETUNREACH | Macro |
A socket operation failed because the subnet containing the remote host was unreachable. |
int ENETRESET | Macro |
A network connection was reset because the remote host crashed. |
int ECONNABORTED | Macro |
A network connection was aborted locally. |
int ECONNRESET | Macro |
A network connection was closed for reasons outside the control of the local host, such as by the remote machine rebooting or an unrecoverable protocol violation. |
int ENOBUFS | Macro |
The kernel's buffers for I/O operations are all in use. In GNU, this
error is always synonymous with ENOMEM ; you may get one or the
other from network operations.
|
int EISCONN | Macro |
You tried to connect a socket that is already connected. See Connecting. |
int ENOTCONN | Macro |
The socket is not connected to anything. You get this error when you
try to transmit data over a socket, without first specifying a
destination for the data. For a connectionless socket (for datagram
protocols, such as UDP), you get EDESTADDRREQ instead.
|
int EDESTADDRREQ | Macro |
No default destination address was set for the socket. You get this
error when you try to transmit data over a connectionless socket,
without first specifying a destination for the data with connect .
|
int ESHUTDOWN | Macro |
The socket has already been shut down. |
int ETOOMANYREFS | Macro |
??? |
int ETIMEDOUT | Macro |
A socket operation with a specified timeout received no response during the timeout period. |
int ECONNREFUSED | Macro |
A remote host refused to allow the network connection (typically because it is not running the requested service). |
int ELOOP | Macro |
Too many levels of symbolic links were encountered in looking up a file name. This often indicates a cycle of symbolic links. |
int ENAMETOOLONG | Macro |
Filename too long (longer than PATH_MAX ; see Limits for Files) or host name too long (in gethostname or
sethostname ; see Host Identification).
|
int EHOSTDOWN | Macro |
The remote host for a requested network connection is down. |
int EHOSTUNREACH | Macro |
The remote host for a requested network connection is not reachable. |
int ENOTEMPTY | Macro |
Directory not empty, where an empty directory was expected. Typically, this error occurs when you are trying to delete a directory. |
int EPROCLIM | Macro |
This means that the per-user limit on new process would be exceeded by
an attempted fork . See Limits on Resources, for details on
the RLIMIT_NPROC limit.
|
int EUSERS | Macro |
The file quota system is confused because there are too many users. |
int EDQUOT | Macro |
The user's disk quota was exceeded. |
int ESTALE | Macro |
Stale NFS file handle. This indicates an internal confusion in the NFS system which is due to file system rearrangements on the server host. Repairing this condition usually requires unmounting and remounting the NFS file system on the local host. |
int EREMOTE | Macro |
An attempt was made to NFS-mount a remote file system with a file name that already specifies an NFS-mounted file. (This is an error on some operating systems, but we expect it to work properly on the GNU system, making this error code impossible.) |
int EBADRPC | Macro |
??? |
int ERPCMISMATCH | Macro |
??? |
int EPROGUNAVAIL | Macro |
??? |
int EPROGMISMATCH | Macro |
??? |
int EPROCUNAVAIL | Macro |
??? |
int ENOLCK | Macro |
No locks available. This is used by the file locking facilities; see File Locks. This error is never generated by the GNU system, but it can result from an operation to an NFS server running another operating system. |
int EFTYPE | Macro |
Inappropriate file type or format. The file was the wrong type for the
operation, or a data file had the wrong format.
On some systems |
int EAUTH | Macro |
??? |
int ENEEDAUTH | Macro |
??? |
int ENOSYS | Macro |
Function not implemented. This indicates that the function called is
not implemented at all, either in the C library itself or in the
operating system. When you get this error, you can be sure that this
particular function will always fail with ENOSYS unless you
install a new version of the C library or the operating system.
|
int ENOTSUP | Macro |
Not supported. A function returns this error when certain parameter
values are valid, but the functionality they request is not available.
This can mean that the function does not implement a particular command
or option value or flag bit at all. For functions that operate on some
object given in a parameter, such as a file descriptor or a port, it
might instead mean that only that specific object (file
descriptor, port, etc.) is unable to support the other parameters given;
different file descriptors might support different ranges of parameter
values.
If the entire function is not available at all in the implementation,
it returns |
int EILSEQ | Macro |
While decoding a multibyte character the function came along an invalid or an incomplete sequence of bytes or the given wide character is invalid. |
int EBACKGROUND | Macro |
In the GNU system, servers supporting the term protocol return
this error for certain operations when the caller is not in the
foreground process group of the terminal. Users do not usually see this
error because functions such as read and write translate
it into a SIGTTIN or SIGTTOU signal. See Job Control,
for information on process groups and these signals.
|
int EDIED | Macro |
In the GNU system, opening a file returns this error when the file is translated by a program and the translator program dies while starting up, before it has connected to the file. |
int ED | Macro |
The experienced user will know what is wrong. |
int EGREGIOUS | Macro |
You did what? |
int EIEIO | Macro |
Go home and have a glass of warm, dairy-fresh milk. |
int EGRATUITOUS | Macro |
This error code has no purpose. |
int EBADMSG | Macro |
int EIDRM | Macro |
int EMULTIHOP | Macro |
int ENODATA | Macro |
int ENOLINK | Macro |
int ENOMSG | Macro |
int ENOSR | Macro |
int ENOSTR | Macro |
int EOVERFLOW | Macro |
int EPROTO | Macro |
int ETIME | Macro |
The following error codes are defined by the Linux/i386 kernel. They are not yet documented.
int ERESTART | Macro |
int ECHRNG | Macro |
int EL2NSYNC | Macro |
int EL3HLT | Macro |
int EL3RST | Macro |
int ELNRNG | Macro |
int EUNATCH | Macro |
int ENOCSI | Macro |
int EL2HLT | Macro |
int EBADE | Macro |
int EBADR | Macro |
int EXFULL | Macro |
int ENOANO | Macro |
int EBADRQC | Macro |
int EBADSLT | Macro |
int EDEADLOCK | Macro |
int EBFONT | Macro |
int ENONET | Macro |
int ENOPKG | Macro |
int EADV | Macro |
int ESRMNT | Macro |
int ECOMM | Macro |
int EDOTDOT | Macro |
int ENOTUNIQ | Macro |
int EBADFD | Macro |
int EREMCHG | Macro |
int ELIBACC | Macro |
int ELIBBAD | Macro |
int ELIBSCN | Macro |
int ELIBMAX | Macro |
int ELIBEXEC | Macro |
int ESTRPIPE | Macro |
int EUCLEAN | Macro |
int ENOTNAM | Macro |
int ENAVAIL | Macro |
int EISNAM | Macro |
int EREMOTEIO | Macro |
int ENOMEDIUM | Macro |
int EMEDIUMTYPE | Macro |
The library has functions and variables designed to make it easy for
your program to report informative error messages in the customary
format about the failure of a library call. The functions
strerror
and perror
give you the standard error message
for a given error code; the variable
program_invocation_short_name
gives you convenient access to the
name of the program that encountered the error.
char * strerror (int errnum) | Function |
The strerror function maps the error code (see Checking for Errors) specified by the errnum argument to a descriptive error
message string. The return value is a pointer to this string.
The value errnum normally comes from the variable You should not modify the string returned by The function |
char * strerror_r (int errnum, char *buf, size_t n) | Function |
The strerror_r function works like strerror but instead of
returning the error message in a statically allocated buffer shared by
all threads in the process, it returns a private copy for the
thread. This might be either some permanent global data or a message
string in the user supplied buffer starting at buf with the
length of n bytes.
At most n characters are written (including the NUL byte) so it is up to the user to select the buffer large enough. This function should always be used in multi-threaded programs since
there is no way to guarantee the string returned by This function |
void perror (const char *message) | Function |
This function prints an error message to the stream stderr ;
see Standard Streams. The orientation of stderr is not
changed.
If you call If you supply a non-null message argument, then The function |
strerror
and perror
produce the exact same message for any
given error code; the precise text varies from system to system. On the
GNU system, the messages are fairly short; there are no multi-line
messages or embedded newlines. Each error message begins with a capital
letter and does not include any terminating punctuation.
Compatibility Note: The strerror
function was introduced
in ISO C89. Many older C systems do not support this function yet.
Many programs that don't read input from the terminal are designed to
exit if any system call fails. By convention, the error message from
such a program should start with the program's name, sans directories.
You can find that name in the variable
program_invocation_short_name
; the full file name is stored the
variable program_invocation_name
.
char * program_invocation_name | Variable |
This variable's value is the name that was used to invoke the program
running in the current process. It is the same as argv[0] . Note
that this is not necessarily a useful file name; often it contains no
directory names. See Program Arguments.
|
char * program_invocation_short_name | Variable |
This variable's value is the name that was used to invoke the program
running in the current process, with directory names removed. (That is
to say, it is the same as program_invocation_name minus
everything up to the last slash, if any.)
|
The library initialization code sets up both of these variables before
calling main
.
Portability Note: These two variables are GNU extensions. If
you want your program to work with non-GNU libraries, you must save the
value of argv[0]
in main
, and then strip off the directory
names yourself. We added these extensions to make it possible to write
self-contained error-reporting subroutines that require no explicit
cooperation from main
.
Here is an example showing how to handle failure to open a file
correctly. The function open_sesame
tries to open the named file
for reading and returns a stream if successful. The fopen
library function returns a null pointer if it couldn't open the file for
some reason. In that situation, open_sesame
constructs an
appropriate error message using the strerror
function, and
terminates the program. If we were going to make some other library
calls before passing the error code to strerror
, we'd have to
save it in a local variable instead, because those other library
functions might overwrite errno
in the meantime.
#include <errno.h> #include <stdio.h> #include <stdlib.h> #include <string.h> FILE * open_sesame (char *name) { FILE *stream; errno = 0; stream = fopen (name, "r"); if (stream == NULL) { fprintf (stderr, "%s: Couldn't open file %s; %s\n", program_invocation_short_name, name, strerror (errno)); exit (EXIT_FAILURE); } else return stream; }
Using perror
has the advantage that the function is portable and
available on all systems implementing ISO C. But often the text
perror
generates is not what is wanted and there is no way to
extend or change what perror
does. The GNU coding standard, for
instance, requires error messages to be preceded by the program name and
programs which read some input files should should provide information
about the input file name and the line number in case an error is
encountered while reading the file. For these occasions there are two
functions available which are widely used throughout the GNU project.
These functions are declared in error.h
.
void error (int status, int errnum, const char *format, ...) | Function |
The error function can be used to report general problems during
program execution. The format argument is a format string just
like those given to the printf family of functions. The
arguments required for the format can follow the format parameter.
Just like perror , error also can report an error code in
textual form. But unlike perror the error value is explicitly
passed to the function in the errnum parameter. This elimintates
the problem mentioned above that the error reporting function must be
called immediately after the function causing the error since otherwise
errno might have a different value.
The The output is directed to the The function will return unless the status parameter has a
non-zero value. In this case the function will call |
void error_at_line (int status, int errnum, const char *fname, unsigned int lineno, const char *format, ...) | Function |
The Directly following the program name a colon, followed by the file name pointer to by fname, another colon, and a value of lineno is printed. This additional output of course is meant to be used to locate an error in an input file (like a programming language source code file etc). If the global variable Just like |
As mentioned above the error
and error_at_line
functions
can be customized by defining a variable named
error_print_progname
.
void (* error_print_progname ) (void) | Variable |
If the error_print_progname variable is defined to a non-zero
value the function pointed to is called by error or
error_at_line . It is expected to print the program name or do
something similarly useful.
The function is expected to be print to the The variable is global and shared by all threads. |
unsigned int error_message_count | Variable |
The error_message_count variable is incremented whenever one of
the functions error or error_at_line returns. The
variable is global and shared by all threads.
|
int error_one_per_line | Variable |
The error_one_per_line variable influences only
error_at_line . Normally the error_at_line function
creates output for every invocation. If error_one_per_line is
set to a non-zero value error_at_line keeps track of the last
file name and line number for which an error was reported and avoid
directly following messages for the same file and line. This variable
is global and shared by all threads.
|
A program which read some input file and reports errors in it could look
like this:
{ char *line = NULL; size_t len = 0; unsigned int lineno = 0; error_message_count = 0; while (! feof_unlocked (fp)) { ssize_t n = getline (&line, &len, fp); if (n <= 0) /* End of file or error. */ break; ++lineno; /* Process the line. */ ... if (Detect error in line) error_at_line (0, errval, filename, lineno, "some error text %s", some_variable); } if (error_message_count != 0) error (EXIT_FAILURE, 0, "%u errors found", error_message_count); }
error
and error_at_line
are clearly the functions of
choice and enable the programmer to write applications which follow the
GNU coding standard. The GNU libc additionally contains functions which
are used in BSD for the same purpose. These functions are declared in
err.h
. It is generally advised to not use these functions. They
are included only for compatibility.
void warn (const char *format, ...) | Function |
The warn function is roughly equivalent to a call like
error (0, errno, format, the parameters) except that the global variables |
void vwarn (const char *format, va_list) | Function |
The vwarn function is just like warn except that the
parameters for the handling of the format string format are passed
in as an value of type va_list .
|
void warnx (const char *format, ...) | Function |
The warnx function is roughly equivalent to a call like
error (0, 0, format, the parameters) except that the global variables |
void vwarnx (const char *format, va_list) | Function |
The vwarnx function is just like warnx except that the
parameters for the handling of the format string format are passed
in as an value of type va_list .
|
void err (int status, const char *format, ...) | Function |
The err function is roughly equivalent to a call like
error (status, errno, format, the parameters) except that the global variables |
void verr (int status, const char *format, va_list) | Function |
The verr function is just like err except that the
parameters for the handling of the format string format are passed
in as an value of type va_list .
|
void errx (int status, const char *format, ...) | Function |
The errx function is roughly equivalent to a call like
error (status, 0, format, the parameters) except that the global variables |
void verrx (int status, const char *format, va_list) | Function |
The verrx function is just like errx except that the
parameters for the handling of the format string format are passed
in as an value of type va_list .
|
This chapter describes how processes manage and use memory in a system that uses the GNU C library.
The GNU C Library has several functions for dynamically allocating virtual memory in various ways. They vary in generality and in efficiency. The library also provides functions for controlling paging and allocation of real memory.
brk
, sbrk
Memory mapped I/O is not discussed in this chapter. See Memory-mapped I/O.
One of the most basic resources a process has available to it is memory. There are a lot of different ways systems organize memory, but in a typical one, each process has one linear virtual address space, with addresses running from zero to some huge maximum. It need not be contiguous; i.e. not all of these addresses actually can be used to store data.
The virtual memory is divided into pages (4 kilobytes is typical). Backing each page of virtual memory is a page of real memory (called a frame) or some secondary storage, usually disk space. The disk space might be swap space or just some ordinary disk file. Actually, a page of all zeroes sometimes has nothing at all backing it - there's just a flag saying it is all zeroes.
The same frame of real memory or backing store can back multiple virtual
pages belonging to multiple processes. This is normally the case, for
example, with virtual memory occupied by GNU C library code. The same
real memory frame containing the printf
function backs a virtual
memory page in each of the existing processes that has a printf
call in its program.
In order for a program to access any part of a virtual page, the page must at that moment be backed by ("connected to") a real frame. But because there is usually a lot more virtual memory than real memory, the pages must move back and forth between real memory and backing store regularly, coming into real memory when a process needs to access them and then retreating to backing store when not needed anymore. This movement is called paging.
When a program attempts to access a page which is not at that moment backed by real memory, this is known as a page fault. When a page fault occurs, the kernel suspends the process, places the page into a real page frame (this is called "paging in" or "faulting in"), then resumes the process so that from the process' point of view, the page was in real memory all along. In fact, to the process, all pages always seem to be in real memory. Except for one thing: the elapsed execution time of an instruction that would normally be a few nanoseconds is suddenly much, much, longer (because the kernel normally has to do I/O to complete the page-in). For programs sensitive to that, the functions described in Locking Pages can control it.
Within each virtual address space, a process has to keep track of what is at which addresses, and that process is called memory allocation. Allocation usually brings to mind meting out scarce resources, but in the case of virtual memory, that's not a major goal, because there is generally much more of it than anyone needs. Memory allocation within a process is mainly just a matter of making sure that the same byte of memory isn't used to store two different things.
Processes allocate memory in two major ways: by exec and programmatically. Actually, forking is a third way, but it's not very interesting. See Creating a Process.
Exec is the operation of creating a virtual address space for a process,
loading its basic program into it, and executing the program. It is
done by the "exec" family of functions (e.g. execl
). The
operation takes a program file (an executable), it allocates space to
load all the data in the executable, loads it, and transfers control to
it. That data is most notably the instructions of the program (the
text), but also literals and constants in the program and even
some variables: C variables with the static storage class (see Memory Allocation and C).
Once that program begins to execute, it uses programmatic allocation to gain additional memory. In a C program with the GNU C library, there are two kinds of programmatic allocation: automatic and dynamic. See Memory Allocation and C.
Memory-mapped I/O is another form of dynamic virtual memory allocation. Mapping memory to a file means declaring that the contents of certain range of a process' addresses shall be identical to the contents of a specified regular file. The system makes the virtual memory initially contain the contents of the file, and if you modify the memory, the system writes the same modification to the file. Note that due to the magic of virtual memory and page faults, there is no reason for the system to do I/O to read the file, or allocate real memory for its contents, until the program accesses the virtual memory. See Memory-mapped I/O.
Just as it programmatically allocates memory, the program can programmatically deallocate (free) it. You can't free the memory that was allocated by exec. When the program exits or execs, you might say that all its memory gets freed, but since in both cases the address space ceases to exist, the point is really moot. See Program Termination.
A process' virtual address space is divided into segments. A segment is a contiguous range of virtual addresses. Three important segments are:
This section covers how ordinary programs manage storage for their data,
including the famous malloc
function and some fancier facilities
special the GNU C library and GNU Compiler.
malloc
facility allows fully general
dynamic allocation.
The C language supports two kinds of memory allocation through the variables in C programs:
In GNU C, the size of the automatic storage can be an expression that varies. In other C implementations, it must be a constant.
A third important kind of memory allocation, dynamic allocation, is not supported by C variables but is available via GNU C library functions.
Dynamic memory allocation is a technique in which programs determine as they are running where to store some information. You need dynamic allocation when the amount of memory you need, or how long you continue to need it, depends on factors that are not known before the program runs.
For example, you may need a block to store a line read from an input file; since there is no limit to how long a line can be, you must allocate the memory dynamically and make it dynamically larger as you read more of the line.
Or, you may need a block for each record or each definition in the input data; since you can't know in advance how many there will be, you must allocate a new block for each record or definition as you read it.
When you use dynamic allocation, the allocation of a block of memory is an action that the program requests explicitly. You call a function or macro when you want to allocate space, and specify the size with an argument. If you want to free the space, you do so by calling another function or macro. You can do these things whenever you want, as often as you want.
Dynamic allocation is not supported by C variables; there is no storage class "dynamic", and there can never be a C variable whose value is stored in dynamically allocated space. The only way to get dynamically allocated memory is via a system call (which is generally via a GNU C library function call), and the only way to refer to dynamically allocated space is through a pointer. Because it is less convenient, and because the actual process of dynamic allocation requires more computation time, programmers generally use dynamic allocation only when neither static nor automatic allocation will serve.
For example, if you want to allocate dynamically some space to hold a
struct foobar
, you cannot declare a variable of type struct
foobar
whose contents are the dynamically allocated space. But you can
declare a variable of pointer type struct foobar *
and assign it the
address of the space. Then you can use the operators *
and
->
on this pointer variable to refer to the contents of the space:
{ struct foobar *ptr = (struct foobar *) malloc (sizeof (struct foobar)); ptr->name = x; ptr->next = current_foobar; current_foobar = ptr; }
The most general dynamic allocation facility is malloc
. It
allows you to allocate blocks of memory of any size at any time, make
them bigger or smaller at any time, and free the blocks individually at
any time (or never).
malloc
.
malloc
. xmalloc
.
free
to free a block you
got with malloc
.
realloc
to make a block
bigger or smaller.
calloc
to allocate a
block and clear it.
mallopt
to adjust allocation
parameters.
malloc
.
malloc
and related functions.
To allocate a block of memory, call malloc
. The prototype for
this function is in stdlib.h
.
void * malloc (size_t size) | Function |
This function returns a pointer to a newly allocated block size bytes long, or a null pointer if the block could not be allocated. |
The contents of the block are undefined; you must initialize it yourself
(or use calloc
instead; see Allocating Cleared Space).
Normally you would cast the value as a pointer to the kind of object
that you want to store in the block. Here we show an example of doing
so, and of initializing the space with zeros using the library function
memset
(see Copying and Concatenation):
struct foo *ptr; ... ptr = (struct foo *) malloc (sizeof (struct foo)); if (ptr == 0) abort (); memset (ptr, 0, sizeof (struct foo));
You can store the result of malloc
into any pointer variable
without a cast, because ISO C automatically converts the type
void *
to another type of pointer when necessary. But the cast
is necessary in contexts other than assignment operators or if you might
want your code to run in traditional C.
Remember that when allocating space for a string, the argument to
malloc
must be one plus the length of the string. This is
because a string is terminated with a null character that doesn't count
in the "length" of the string but does need space. For example:
char *ptr; ... ptr = (char *) malloc (length + 1);
See Representation of Strings, for more information about this.
malloc
If no more space is available, malloc
returns a null pointer.
You should check the value of every call to malloc
. It is
useful to write a subroutine that calls malloc
and reports an
error if the value is a null pointer, returning only if the value is
nonzero. This function is conventionally called xmalloc
. Here
it is:
void * xmalloc (size_t size) { register void *value = malloc (size); if (value == 0) fatal ("virtual memory exhausted"); return value; }
Here is a real example of using malloc
(by way of xmalloc
).
The function savestring
will copy a sequence of characters into
a newly allocated null-terminated string:
char * savestring (const char *ptr, size_t len) { register char *value = (char *) xmalloc (len + 1); value[len] = '\0'; return (char *) memcpy (value, ptr, len); }
The block that malloc
gives you is guaranteed to be aligned so
that it can hold any type of data. In the GNU system, the address is
always a multiple of eight on most systems, and a multiple of 16 on
64-bit systems. Only rarely is any higher boundary (such as a page
boundary) necessary; for those cases, use memalign
,
posix_memalign
or valloc
(see Aligned Memory Blocks).
Note that the memory located after the end of the block is likely to be
in use for something else; perhaps a block already allocated by another
call to malloc
. If you attempt to treat the block as longer than
you asked for it to be, you are liable to destroy the data that
malloc
uses to keep track of its blocks, or you may destroy the
contents of another block. If you have already allocated a block and
discover you want it to be bigger, use realloc
(see Changing Block Size).
malloc
When you no longer need a block that you got with malloc
, use the
function free
to make the block available to be allocated again.
The prototype for this function is in stdlib.h
.
void free (void *ptr) | Function |
The free function deallocates the block of memory pointed at
by ptr.
|
void cfree (void *ptr) | Function |
This function does the same thing as free . It's provided for
backward compatibility with SunOS; you should use free instead.
|
Freeing a block alters the contents of the block. Do not expect to
find any data (such as a pointer to the next block in a chain of blocks) in
the block after freeing it. Copy whatever you need out of the block before
freeing it! Here is an example of the proper way to free all the blocks in
a chain, and the strings that they point to:
struct chain { struct chain *next; char *name; } void free_chain (struct chain *chain) { while (chain != 0) { struct chain *next = chain->next; free (chain->name); free (chain); chain = next; } }
Occasionally, free
can actually return memory to the operating
system and make the process smaller. Usually, all it can do is allow a
later call to malloc
to reuse the space. In the meantime, the
space remains in your program as part of a free-list used internally by
malloc
.
There is no point in freeing blocks at the end of a program, because all of the program's space is given back to the system when the process terminates.
Often you do not know for certain how big a block you will ultimately need at the time you must begin to use the block. For example, the block might be a buffer that you use to hold a line being read from a file; no matter how long you make the buffer initially, you may encounter a line that is longer.
You can make the block longer by calling realloc
. This function
is declared in stdlib.h
.
void * realloc (void *ptr, size_t newsize) | Function |
The realloc function changes the size of the block whose address is
ptr to be newsize.
Since the space after the end of the block may be in use, If you pass a null pointer for ptr, |
Like malloc
, realloc
may return a null pointer if no
memory space is available to make the block bigger. When this happens,
the original block is untouched; it has not been modified or relocated.
In most cases it makes no difference what happens to the original block
when realloc
fails, because the application program cannot continue
when it is out of memory, and the only thing to do is to give a fatal error
message. Often it is convenient to write and use a subroutine,
conventionally called xrealloc
, that takes care of the error message
as xmalloc
does for malloc
:
void * xrealloc (void *ptr, size_t size) { register void *value = realloc (ptr, size); if (value == 0) fatal ("Virtual memory exhausted"); return value; }
You can also use realloc
to make a block smaller. The reason you
would do this is to avoid tying up a lot of memory space when only a little
is needed.
In several allocation implementations, making a block smaller sometimes
necessitates copying it, so it can fail if no other space is available.
If the new size you specify is the same as the old size, realloc
is guaranteed to change nothing and return the same address that you gave.
The function calloc
allocates memory and clears it to zero. It
is declared in stdlib.h
.
void * calloc (size_t count, size_t eltsize) | Function |
This function allocates a block long enough to contain a vector of
count elements, each of size eltsize. Its contents are
cleared to zero before calloc returns.
|
You could define calloc
as follows:
void * calloc (size_t count, size_t eltsize) { size_t size = count * eltsize; void *value = malloc (size); if (value != 0) memset (value, 0, size); return value; }
But in general, it is not guaranteed that calloc
calls
malloc
internally. Therefore, if an application provides its own
malloc
/realloc
/free
outside the C library, it
should always define calloc
, too.
malloc
As opposed to other versions, the malloc
in the GNU C Library
does not round up block sizes to powers of two, neither for large nor
for small sizes. Neighboring chunks can be coalesced on a free
no matter what their size is. This makes the implementation suitable
for all kinds of allocation patterns without generally incurring high
memory waste through fragmentation.
Very large blocks (much larger than a page) are allocated with
mmap
(anonymous or via /dev/zero
) by this implementation.
This has the great advantage that these chunks are returned to the
system immediately when they are freed. Therefore, it cannot happen
that a large chunk becomes "locked" in between smaller ones and even
after calling free
wastes memory. The size threshold for
mmap
to be used can be adjusted with mallopt
. The use of
mmap
can also be disabled completely.
The address of a block returned by malloc
or realloc
in
the GNU system is always a multiple of eight (or sixteen on 64-bit
systems). If you need a block whose address is a multiple of a higher
power of two than that, use memalign
, posix_memalign
, or
valloc
. These functions are declared in stdlib.h
.
With the GNU library, you can use free
to free the blocks that
memalign
, posix_memalign
, and valloc
return. That
does not work in BSD, however--BSD does not provide any way to free
such blocks.
void * memalign (size_t boundary, size_t size) | Function |
The memalign function allocates a block of size bytes whose
address is a multiple of boundary. The boundary must be a
power of two! The function memalign works by allocating a
somewhat larger block, and then returning an address within the block
that is on the specified boundary.
|
int posix_memalign (void **memptr, size_t alignment, size_t size) | Function |
The posix_memalign function is similar to the memalign
function in that it returns a buffer of size bytes aligned to a
multiple of alignment. But it adds one requirement to the
parameter alignment: the value must be a power of two multiple of
sizeof (void *) .
If the function succeeds in allocation memory a pointer to the allocated
memory is returned in This function was introduced in POSIX 1003.1d. |
void * valloc (size_t size) | Function |
Using valloc is like using memalign and passing the page size
as the value of the second argument. It is implemented like this:
void * valloc (size_t size) { return memalign (getpagesize (), size); }Query Memory Parameters for more information about the memory subsystem. |
You can adjust some parameters for dynamic memory allocation with the
mallopt
function. This function is the general SVID/XPG
interface, defined in malloc.h
.
int mallopt (int param, int value) | Function |
When calling mallopt , the param argument specifies the
parameter to be set, and value the new value to be set. Possible
choices for param, as defined in malloc.h , are:
|
You can ask malloc
to check the consistency of dynamic memory by
using the mcheck
function. This function is a GNU extension,
declared in mcheck.h
.
int mcheck (void (*abortfn) (enum mcheck_status status)) | Function |
Calling mcheck tells malloc to perform occasional
consistency checks. These will catch things such as writing
past the end of a block that was allocated with malloc .
The abortfn argument is the function to call when an inconsistency
is found. If you supply a null pointer, then It is too late to begin allocation checking once you have allocated
anything with The easiest way to arrange to call (gdb) break main Breakpoint 1, main (argc=2, argv=0xbffff964) at whatever.c:10 (gdb) command 1 Type commands for when breakpoint 1 is hit, one per line. End with a line saying just "end". >call mcheck(0) >continue >end (gdb) ... This will however only work if no initialization function of any object
involved calls any of the |
enum mcheck_status mprobe (void *pointer) | Function |
The mprobe function lets you explicitly check for inconsistencies
in a particular allocated block. You must have already called
mcheck at the beginning of the program, to do its occasional
checks; calling mprobe requests an additional consistency check
to be done at the time of the call.
The argument pointer must be a pointer returned by |
enum mcheck_status | Data Type |
This enumerated type describes what kind of inconsistency was detected
in an allocated block, if any. Here are the possible values:
|
Another possibility to check for and guard against bugs in the use of
malloc
, realloc
and free
is to set the environment
variable MALLOC_CHECK_
. When MALLOC_CHECK_
is set, a
special (less efficient) implementation is used which is designed to be
tolerant against simple errors, such as double calls of free
with
the same argument, or overruns of a single byte (off-by-one bugs). Not
all such errors can be protected against, however, and memory leaks can
result. If MALLOC_CHECK_
is set to 0
, any detected heap
corruption is silently ignored; if set to 1
, a diagnostic is
printed on stderr
; if set to 2
, abort
is called
immediately. This can be useful because otherwise a crash may happen
much later, and the true cause for the problem is then very hard to
track down.
There is one problem with MALLOC_CHECK_
: in SUID or SGID binaries
it could possibly be exploited since diverging from the normal programs
behavior it now writes something to the standard error descriptor.
Therefore the use of MALLOC_CHECK_
is disabled by default for
SUID and SGID binaries. It can be enabled again by the system
administrator by adding a file /etc/suid-debug
(the content is
not important it could be empty).
So, what's the difference between using MALLOC_CHECK_
and linking
with -lmcheck
? MALLOC_CHECK_
is orthogonal with respect to
-lmcheck
. -lmcheck
has been added for backward
compatibility. Both MALLOC_CHECK_
and -lmcheck
should
uncover the same bugs - but using MALLOC_CHECK_
you don't need to
recompile your application.
The GNU C library lets you modify the behavior of malloc
,
realloc
, and free
by specifying appropriate hook
functions. You can use these hooks to help you debug programs that use
dynamic memory allocation, for example.
The hook variables are declared in malloc.h
.
__malloc_hook | Variable |
The value of this variable is a pointer to the function that
malloc uses whenever it is called. You should define this
function to look like malloc ; that is, like:
void *function (size_t size, const void *caller) The value of caller is the return address found on the stack when
the |
__realloc_hook | Variable |
The value of this variable is a pointer to function that realloc
uses whenever it is called. You should define this function to look
like realloc ; that is, like:
void *function (void *ptr, size_t size, const void *caller) The value of caller is the return address found on the stack when
the |
__free_hook | Variable |
The value of this variable is a pointer to function that free
uses whenever it is called. You should define this function to look
like free ; that is, like:
void function (void *ptr, const void *caller) The value of caller is the return address found on the stack when
the |
__memalign_hook | Variable |
The value of this variable is a pointer to function that memalign
uses whenever it is called. You should define this function to look
like memalign ; that is, like:
void *function (size_t size, size_t alignment, const void *caller) The value of caller is the return address found on the stack when
the |
You must make sure that the function you install as a hook for one of these functions does not call that function recursively without restoring the old value of the hook first! Otherwise, your program will get stuck in an infinite recursion. Before calling the function recursively, one should make sure to restore all the hooks to their previous value. When coming back from the recursive call, all the hooks should be resaved since a hook might modify itself.
__malloc_initialize_hook | Variable |
The value of this variable is a pointer to a function that is called
once when the malloc implementation is initialized. This is a weak
variable, so it can be overridden in the application with a definition
like the following:
void (*__malloc_initialize_hook) (void) = my_init_hook; |
An issue to look out for is the time at which the malloc hook functions
can be safely installed. If the hook functions call the malloc-related
functions recursively, it is necessary that malloc has already properly
initialized itself at the time when __malloc_hook
etc. is
assigned to. On the other hand, if the hook functions provide a
complete malloc implementation of their own, it is vital that the hooks
are assigned to before the very first malloc
call has
completed, because otherwise a chunk obtained from the ordinary,
un-hooked malloc may later be handed to __free_hook
, for example.
In both cases, the problem can be solved by setting up the hooks from
within a user-defined function pointed to by
__malloc_initialize_hook
--then the hooks will be set up safely
at the right time.
Here is an example showing how to use __malloc_hook
and
__free_hook
properly. It installs a function that prints out
information every time malloc
or free
is called. We just
assume here that realloc
and memalign
are not used in our
program.
/* Prototypes for __malloc_hook, __free_hook */ #include <malloc.h> /* Prototypes for our hooks. */ static void *my_init_hook (void); static void *my_malloc_hook (size_t, const void *); static void my_free_hook (void*, const void *); /* Override initializing hook from the C library. */ void (*__malloc_initialize_hook) (void) = my_init_hook; static void my_init_hook (void) { old_malloc_hook = __malloc_hook; old_free_hook = __free_hook; __malloc_hook = my_malloc_hook; __free_hook = my_free_hook; } static void * my_malloc_hook (size_t size, const void *caller) { void *result; /* Restore all old hooks */ __malloc_hook = old_malloc_hook; __free_hook = old_free_hook; /* Call recursively */ result = malloc (size); /* Save underlying hooks */ old_malloc_hook = __malloc_hook; old_free_hook = __free_hook; /*printf
might callmalloc
, so protect it too. */ printf ("malloc (%u) returns %p\n", (unsigned int) size, result); /* Restore our own hooks */ __malloc_hook = my_malloc_hook; __free_hook = my_free_hook; return result; } static void * my_free_hook (void *ptr, const void *caller) { /* Restore all old hooks */ __malloc_hook = old_malloc_hook; __free_hook = old_free_hook; /* Call recursively */ free (ptr); /* Save underlying hooks */ old_malloc_hook = __malloc_hook; old_free_hook = __free_hook; /*printf
might callfree
, so protect it too. */ printf ("freed pointer %p\n", ptr); /* Restore our own hooks */ __malloc_hook = my_malloc_hook; __free_hook = my_free_hook; } main () { ... }
The mcheck
function (see Heap Consistency Checking) works by
installing such hooks.
malloc
You can get information about dynamic memory allocation by calling the
mallinfo
function. This function and its associated data type
are declared in malloc.h
; they are an extension of the standard
SVID/XPG version.
struct mallinfo | Data Type |
This structure type is used to return information about the dynamic
memory allocator. It contains the following members:
|
struct mallinfo mallinfo (void) | Function |
This function returns information about the current dynamic memory usage
in a structure of type struct mallinfo .
|
malloc
-Related FunctionsHere is a summary of the functions that work with malloc
:
void *malloc (size_t size)
void free (void *addr)
malloc
. See Freeing after Malloc.
void *realloc (void *addr, size_t size)
malloc
larger or smaller,
possibly by copying it to a new location. See Changing Block Size.
void *calloc (size_t count, size_t eltsize)
malloc
, and set its contents to zero. See Allocating Cleared Space.
void *valloc (size_t size)
void *memalign (size_t size, size_t boundary)
int mallopt (int param, int value)
int mcheck (void (*abortfn) (void))
malloc
to perform occasional consistency checks on
dynamically allocated memory, and to call abortfn when an
inconsistency is found. See Heap Consistency Checking.
void *(*__malloc_hook) (size_t size, const void *caller)
malloc
uses whenever it is called.
void *(*__realloc_hook) (void *ptr, size_t size, const void *caller)
realloc
uses whenever it is called.
void (*__free_hook) (void *ptr, const void *caller)
free
uses whenever it is called.
void (*__memalign_hook) (size_t size, size_t alignment, const void *caller)
memalign
uses whenever it is called.
struct mallinfo mallinfo (void)
A complicated task when programming with languages which do not use garbage collected dynamic memory allocation is to find memory leaks. Long running programs must assure that dynamically allocated objects are freed at the end of their lifetime. If this does not happen the system runs out of memory, sooner or later.
The malloc
implementation in the GNU C library provides some
simple means to detect such leaks and obtain some information to find
the location. To do this the application must be started in a special
mode which is enabled by an environment variable. There are no speed
penalties for the program if the debugging mode is not enabled.
void mtrace (void) | Function |
When the mtrace function is called it looks for an environment
variable named MALLOC_TRACE . This variable is supposed to
contain a valid file name. The user must have write access. If the
file already exists it is truncated. If the environment variable is not
set or it does not name a valid file which can be opened for writing
nothing is done. The behavior of malloc etc. is not changed.
For obvious reasons this also happens if the application is installed
with the SUID or SGID bit set.
If the named file is successfully opened, This function is a GNU extension and generally not available on other
systems. The prototype can be found in |
void muntrace (void) | Function |
The muntrace function can be called after mtrace was used
to enable tracing the malloc calls. If no (successful) call of
mtrace was made muntrace does nothing.
Otherwise it deinstalls the handlers for This function is a GNU extension and generally not available on other
systems. The prototype can be found in |
Even though the tracing functionality does not influence the runtime
behavior of the program it is not a good idea to call mtrace
in
all programs. Just imagine that you debug a program using mtrace
and all other programs used in the debugging session also trace their
malloc
calls. The output file would be the same for all programs
and thus is unusable. Therefore one should call mtrace
only if
compiled for debugging. A program could therefore start like this:
#include <mcheck.h> int main (int argc, char *argv[]) { #ifdef DEBUGGING mtrace (); #endif ... }
This is all what is needed if you want to trace the calls during the
whole runtime of the program. Alternatively you can stop the tracing at
any time with a call to muntrace
. It is even possible to restart
the tracing again with a new call to mtrace
. But this can cause
unreliable results since there may be calls of the functions which are
not called. Please note that not only the application uses the traced
functions, also libraries (including the C library itself) use these
functions.
This last point is also why it is no good idea to call muntrace
before the program terminated. The libraries are informed about the
termination of the program only after the program returns from
main
or calls exit
and so cannot free the memory they use
before this time.
So the best thing one can do is to call mtrace
as the very first
function in the program and never call muntrace
. So the program
traces almost all uses of the malloc
functions (except those
calls which are executed by constructors of the program or used
libraries).
You know the situation. The program is prepared for debugging and in
all debugging sessions it runs well. But once it is started without
debugging the error shows up. A typical example is a memory leak that
becomes visible only when we turn off the debugging. If you foresee
such situations you can still win. Simply use something equivalent to
the following little program:
#include <mcheck.h> #include <signal.h> static void enable (int sig) { mtrace (); signal (SIGUSR1, enable); } static void disable (int sig) { muntrace (); signal (SIGUSR2, disable); } int main (int argc, char *argv[]) { ... signal (SIGUSR1, enable); signal (SIGUSR2, disable); ... }
I.e., the user can start the memory debugger any time s/he wants if the
program was started with MALLOC_TRACE
set in the environment.
The output will of course not show the allocations which happened before
the first signal but if there is a memory leak this will show up
nevertheless.
If you take a look at the output it will look similar to this:
= Start [0x8048209] - 0x8064cc8 [0x8048209] - 0x8064ce0 [0x8048209] - 0x8064cf8 [0x80481eb] + 0x8064c48 0x14 [0x80481eb] + 0x8064c60 0x14 [0x80481eb] + 0x8064c78 0x14 [0x80481eb] + 0x8064c90 0x14 = End
What this all means is not really important since the trace file is not
meant to be read by a human. Therefore no attention is given to
readability. Instead there is a program which comes with the GNU C
library which interprets the traces and outputs a summary in an
user-friendly way. The program is called mtrace
(it is in fact a
Perl script) and it takes one or two arguments. In any case the name of
the file with the trace output must be specified. If an optional
argument precedes the name of the trace file this must be the name of
the program which generated the trace.
drepper$ mtrace tst-mtrace log No memory leaks.
In this case the program tst-mtrace
was run and it produced a
trace file log
. The message printed by mtrace
shows there
are no problems with the code, all allocated memory was freed
afterwards.
If we call mtrace
on the example trace given above we would get a
different outout:
drepper$ mtrace errlog - 0x08064cc8 Free 2 was never alloc'd 0x8048209 - 0x08064ce0 Free 3 was never alloc'd 0x8048209 - 0x08064cf8 Free 4 was never alloc'd 0x8048209 Memory not freed: ----------------- Address Size Caller 0x08064c48 0x14 at 0x80481eb 0x08064c60 0x14 at 0x80481eb 0x08064c78 0x14 at 0x80481eb 0x08064c90 0x14 at 0x80481eb
We have called mtrace
with only one argument and so the script
has no chance to find out what is meant with the addresses given in the
trace. We can do better:
drepper$ mtrace tst errlog - 0x08064cc8 Free 2 was never alloc'd /home/drepper/tst.c:39 - 0x08064ce0 Free 3 was never alloc'd /home/drepper/tst.c:39 - 0x08064cf8 Free 4 was never alloc'd /home/drepper/tst.c:39 Memory not freed: ----------------- Address Size Caller 0x08064c48 0x14 at /home/drepper/tst.c:33 0x08064c60 0x14 at /home/drepper/tst.c:33 0x08064c78 0x14 at /home/drepper/tst.c:33 0x08064c90 0x14 at /home/drepper/tst.c:33
Suddenly the output makes much more sense and the user can see immediately where the function calls causing the trouble can be found.
Interpreting this output is not complicated. There are at most two
different situations being detected. First, free
was called for
pointers which were never returned by one of the allocation functions.
This is usually a very bad problem and what this looks like is shown in
the first three lines of the output. Situations like this are quite
rare and if they appear they show up very drastically: the program
normally crashes.
The other situation which is much harder to detect are memory leaks. As
you can see in the output the mtrace
function collects all this
information and so can say that the program calls an allocation function
from line 33 in the source file /home/drepper/tst-mtrace.c
four
times without freeing this memory before the program terminates.
Whether this is a real problem remains to be investigated.
An obstack is a pool of memory containing a stack of objects. You can create any number of separate obstacks, and then allocate objects in specified obstacks. Within each obstack, the last object allocated must always be the first one freed, but distinct obstacks are independent of each other.
Aside from this one constraint of order of freeing, obstacks are totally general: an obstack can contain any number of objects of any size. They are implemented with macros, so allocation is usually very fast as long as the objects are usually small. And the only space overhead per object is the padding needed to start each object on a suitable boundary.
The utilities for manipulating obstacks are declared in the header
file obstack.h
.
struct obstack | Data Type |
An obstack is represented by a data structure of type struct
obstack . This structure has a small fixed size; it records the status
of the obstack and how to find the space in which objects are allocated.
It does not contain any of the objects themselves. You should not try
to access the contents of the structure directly; use only the functions
described in this chapter.
|
You can declare variables of type struct obstack
and use them as
obstacks, or you can allocate obstacks dynamically like any other kind
of object. Dynamic allocation of obstacks allows your program to have a
variable number of different stacks. (You can even allocate an
obstack structure in another obstack, but this is rarely useful.)
All the functions that work with obstacks require you to specify which
obstack to use. You do this with a pointer of type struct obstack
*
. In the following, we often say "an obstack" when strictly
speaking the object at hand is such a pointer.
The objects in the obstack are packed into large blocks called
chunks. The struct obstack
structure points to a chain of
the chunks currently in use.
The obstack library obtains a new chunk whenever you allocate an object
that won't fit in the previous chunk. Since the obstack library manages
chunks automatically, you don't need to pay much attention to them, but
you do need to supply a function which the obstack library should use to
get a chunk. Usually you supply a function which uses malloc
directly or indirectly. You must also supply a function to free a chunk.
These matters are described in the following section.
Each source file in which you plan to use the obstack functions
must include the header file obstack.h
, like this:
#include <obstack.h>
Also, if the source file uses the macro obstack_init
, it must
declare or define two functions or macros that will be called by the
obstack library. One, obstack_chunk_alloc
, is used to allocate
the chunks of memory into which objects are packed. The other,
obstack_chunk_free
, is used to return chunks when the objects in
them are freed. These macros should appear before any use of obstacks
in the source file.
Usually these are defined to use malloc
via the intermediary
xmalloc
(see Unconstrained Allocation). This is done with
the following pair of macro definitions:
#define obstack_chunk_alloc xmalloc #define obstack_chunk_free free
Though the memory you get using obstacks really comes from malloc
,
using obstacks is faster because malloc
is called less often, for
larger blocks of memory. See Obstack Chunks, for full details.
At run time, before the program can use a struct obstack
object
as an obstack, it must initialize the obstack by calling
obstack_init
.
int obstack_init (struct obstack *obstack-ptr) | Function |
Initialize obstack obstack-ptr for allocation of objects. This
function calls the obstack's obstack_chunk_alloc function. If
allocation of memory fails, the function pointed to by
obstack_alloc_failed_handler is called. The obstack_init
function always returns 1 (Compatibility notice: Former versions of
obstack returned 0 if allocation failed).
|
Here are two examples of how to allocate the space for an obstack and
initialize it. First, an obstack that is a static variable:
static struct obstack myobstack; ... obstack_init (&myobstack);
Second, an obstack that is itself dynamically allocated:
struct obstack *myobstack_ptr = (struct obstack *) xmalloc (sizeof (struct obstack)); obstack_init (myobstack_ptr);
obstack_alloc_failed_handler | Variable |
The value of this variable is a pointer to a function that
obstack uses when obstack_chunk_alloc fails to allocate
memory. The default action is to print a message and abort.
You should supply a function that either calls exit
(see Program Termination) or longjmp (see Non-Local Exits) and doesn't return.
void my_obstack_alloc_failed (void) ... obstack_alloc_failed_handler = &my_obstack_alloc_failed; |
The most direct way to allocate an object in an obstack is with
obstack_alloc
, which is invoked almost like malloc
.
void * obstack_alloc (struct obstack *obstack-ptr, int size) | Function |
This allocates an uninitialized block of size bytes in an obstack
and returns its address. Here obstack-ptr specifies which obstack
to allocate the block in; it is the address of the struct obstack
object which represents the obstack. Each obstack function or macro
requires you to specify an obstack-ptr as the first argument.
This function calls the obstack's |
For example, here is a function that allocates a copy of a string str
in a specific obstack, which is in the variable string_obstack
:
struct obstack string_obstack; char * copystring (char *string) { size_t len = strlen (string) + 1; char *s = (char *) obstack_alloc (&string_obstack, len); memcpy (s, string, len); return s; }
To allocate a block with specified contents, use the function
obstack_copy
, declared like this:
void * obstack_copy (struct obstack *obstack-ptr, void *address, int size) | Function |
This allocates a block and initializes it by copying size
bytes of data starting at address. It calls
obstack_alloc_failed_handler if allocation of memory by
obstack_chunk_alloc failed.
|
void * obstack_copy0 (struct obstack *obstack-ptr, void *address, int size) | Function |
Like obstack_copy , but appends an extra byte containing a null
character. This extra byte is not counted in the argument size.
|
The obstack_copy0
function is convenient for copying a sequence
of characters into an obstack as a null-terminated string. Here is an
example of its use:
char * obstack_savestring (char *addr, int size) { return obstack_copy0 (&myobstack, addr, size); }
Contrast this with the previous example of savestring
using
malloc
(see Basic Allocation).
To free an object allocated in an obstack, use the function
obstack_free
. Since the obstack is a stack of objects, freeing
one object automatically frees all other objects allocated more recently
in the same obstack.
void obstack_free (struct obstack *obstack-ptr, void *object) | Function |
If object is a null pointer, everything allocated in the obstack is freed. Otherwise, object must be the address of an object allocated in the obstack. Then object is freed, along with everything allocated in obstack since object. |
Note that if object is a null pointer, the result is an
uninitialized obstack. To free all memory in an obstack but leave it
valid for further allocation, call obstack_free
with the address
of the first object allocated on the obstack:
obstack_free (obstack_ptr, first_object_allocated_ptr);
Recall that the objects in an obstack are grouped into chunks. When all the objects in a chunk become free, the obstack library automatically frees the chunk (see Preparing for Obstacks). Then other obstacks, or non-obstack allocation, can reuse the space of the chunk.
The interfaces for using obstacks may be defined either as functions or as macros, depending on the compiler. The obstack facility works with all C compilers, including both ISO C and traditional C, but there are precautions you must take if you plan to use compilers other than GNU C.
If you are using an old-fashioned non-ISO C compiler, all the obstack "functions" are actually defined only as macros. You can call these macros like functions, but you cannot use them in any other way (for example, you cannot take their address).
Calling the macros requires a special precaution: namely, the first
operand (the obstack pointer) may not contain any side effects, because
it may be computed more than once. For example, if you write this:
obstack_alloc (get_obstack (), 4);
you will find that get_obstack
may be called several times.
If you use *obstack_list_ptr++
as the obstack pointer argument,
you will get very strange results since the incrementation may occur
several times.
In ISO C, each function has both a macro definition and a function
definition. The function definition is used if you take the address of the
function without calling it. An ordinary call uses the macro definition by
default, but you can request the function definition instead by writing the
function name in parentheses, as shown here:
char *x; void *(*funcp) (); /* Use the macro. */ x = (char *) obstack_alloc (obptr, size); /* Call the function. */ x = (char *) (obstack_alloc) (obptr, size); /* Take the address of the function. */ funcp = obstack_alloc;
This is the same situation that exists in ISO C for the standard library functions. See Macro Definitions.
Warning: When you do use the macros, you must observe the precaution of avoiding side effects in the first operand, even in ISO C.
If you use the GNU C compiler, this precaution is not necessary, because various language extensions in GNU C permit defining the macros so as to compute each argument only once.
Because memory in obstack chunks is used sequentially, it is possible to build up an object step by step, adding one or more bytes at a time to the end of the object. With this technique, you do not need to know how much data you will put in the object until you come to the end of it. We call this the technique of growing objects. The special functions for adding data to the growing object are described in this section.
You don't need to do anything special when you start to grow an object.
Using one of the functions to add data to the object automatically
starts it. However, it is necessary to say explicitly when the object is
finished. This is done with the function obstack_finish
.
The actual address of the object thus built up is not known until the object is finished. Until then, it always remains possible that you will add so much data that the object must be copied into a new chunk.
While the obstack is in use for a growing object, you cannot use it for ordinary allocation of another object. If you try to do so, the space already added to the growing object will become part of the other object.
void obstack_blank (struct obstack *obstack-ptr, int size) | Function |
The most basic function for adding to a growing object is
obstack_blank , which adds space without initializing it.
|
void obstack_grow (struct obstack *obstack-ptr, void *data, int size) | Function |
To add a block of initialized space, use obstack_grow , which is
the growing-object analogue of obstack_copy . It adds size
bytes of data to the growing object, copying the contents from
data.
|
void obstack_grow0 (struct obstack *obstack-ptr, void *data, int size) | Function |
This is the growing-object analogue of obstack_copy0 . It adds
size bytes copied from data, followed by an additional null
character.
|
void obstack_1grow (struct obstack *obstack-ptr, char c) | Function |
To add one character at a time, use the function obstack_1grow .
It adds a single byte containing c to the growing object.
|
void obstack_ptr_grow (struct obstack *obstack-ptr, void *data) | Function |
Adding the value of a pointer one can use the function
obstack_ptr_grow . It adds sizeof (void *) bytes
containing the value of data.
|
void obstack_int_grow (struct obstack *obstack-ptr, int data) | Function |
A single value of type int can be added by using the
obstack_int_grow function. It adds sizeof (int) bytes to
the growing object and initializes them with the value of data.
|
void * obstack_finish (struct obstack *obstack-ptr) | Function |
When you are finished growing the object, use the function
obstack_finish to close it off and return its final address.
Once you have finished the object, the obstack is available for ordinary allocation or for growing another object. This function can return a null pointer under the same conditions as
|
When you build an object by growing it, you will probably need to know
afterward how long it became. You need not keep track of this as you grow
the object, because you can find out the length from the obstack just
before finishing the object with the function obstack_object_size
,
declared as follows:
int obstack_object_size (struct obstack *obstack-ptr) | Function |
This function returns the current size of the growing object, in bytes.
Remember to call this function before finishing the object.
After it is finished, obstack_object_size will return zero.
|
If you have started growing an object and wish to cancel it, you should
finish it and then free it, like this:
obstack_free (obstack_ptr, obstack_finish (obstack_ptr));
This has no effect if no object was growing.
You can use obstack_blank
with a negative size argument to make
the current object smaller. Just don't try to shrink it beyond zero
length--there's no telling what will happen if you do that.
The usual functions for growing objects incur overhead for checking whether there is room for the new growth in the current chunk. If you are frequently constructing objects in small steps of growth, this overhead can be significant.
You can reduce the overhead by using special "fast growth" functions that grow the object without checking. In order to have a robust program, you must do the checking yourself. If you do this checking in the simplest way each time you are about to add data to the object, you have not saved anything, because that is what the ordinary growth functions do. But if you can arrange to check less often, or check more efficiently, then you make the program faster.
The function obstack_room
returns the amount of room available
in the current chunk. It is declared as follows:
int obstack_room (struct obstack *obstack-ptr) | Function |
This returns the number of bytes that can be added safely to the current growing object (or to an object about to be started) in obstack obstack using the fast growth functions. |
While you know there is room, you can use these fast growth functions for adding data to a growing object:
void obstack_1grow_fast (struct obstack *obstack-ptr, char c) | Function |
The function obstack_1grow_fast adds one byte containing the
character c to the growing object in obstack obstack-ptr.
|
void obstack_ptr_grow_fast (struct obstack *obstack-ptr, void *data) | Function |
The function obstack_ptr_grow_fast adds sizeof (void *)
bytes containing the value of data to the growing object in
obstack obstack-ptr.
|
void obstack_int_grow_fast (struct obstack *obstack-ptr, int data) | Function |
The function obstack_int_grow_fast adds sizeof (int) bytes
containing the value of data to the growing object in obstack
obstack-ptr.
|
void obstack_blank_fast (struct obstack *obstack-ptr, int size) | Function |
The function obstack_blank_fast adds size bytes to the
growing object in obstack obstack-ptr without initializing them.
|
When you check for space using obstack_room
and there is not
enough room for what you want to add, the fast growth functions
are not safe. In this case, simply use the corresponding ordinary
growth function instead. Very soon this will copy the object to a
new chunk; then there will be lots of room available again.
So, each time you use an ordinary growth function, check afterward for
sufficient space using obstack_room
. Once the object is copied
to a new chunk, there will be plenty of space again, so the program will
start using the fast growth functions again.
Here is an example:
void add_string (struct obstack *obstack, const char *ptr, int len) { while (len > 0) { int room = obstack_room (obstack); if (room == 0) { /* Not enough room. Add one character slowly, which may copy to a new chunk and make room. */ obstack_1grow (obstack, *ptr++); len--; } else { if (room > len) room = len; /* Add fast as much as we have room for. */ len -= room; while (room-- > 0) obstack_1grow_fast (obstack, *ptr++); } } }
Here are functions that provide information on the current status of allocation in an obstack. You can use them to learn about an object while still growing it.
void * obstack_base (struct obstack *obstack-ptr) | Function |
This function returns the tentative address of the beginning of the
currently growing object in obstack-ptr. If you finish the object
immediately, it will have that address. If you make it larger first, it
may outgrow the current chunk--then its address will change!
If no object is growing, this value says where the next object you allocate will start (once again assuming it fits in the current chunk). |
void * obstack_next_free (struct obstack *obstack-ptr) | Function |
This function returns the address of the first free byte in the current
chunk of obstack obstack-ptr. This is the end of the currently
growing object. If no object is growing, obstack_next_free
returns the same value as obstack_base .
|
int obstack_object_size (struct obstack *obstack-ptr) | Function |
This function returns the size in bytes of the currently growing object.
This is equivalent to
obstack_next_free (obstack-ptr) - obstack_base (obstack-ptr) |
Each obstack has an alignment boundary; each object allocated in the obstack automatically starts on an address that is a multiple of the specified boundary. By default, this boundary is 4 bytes.
To access an obstack's alignment boundary, use the macro
obstack_alignment_mask
, whose function prototype looks like
this:
int obstack_alignment_mask (struct obstack *obstack-ptr) | Macro |
The value is a bit mask; a bit that is 1 indicates that the corresponding
bit in the address of an object should be 0. The mask value should be one
less than a power of 2; the effect is that all object addresses are
multiples of that power of 2. The default value of the mask is 3, so that
addresses are multiples of 4. A mask value of 0 means an object can start
on any multiple of 1 (that is, no alignment is required).
The expansion of the macro obstack_alignment_mask (obstack_ptr) = 0; has the effect of turning off alignment processing in the specified obstack. |
Note that a change in alignment mask does not take effect until
after the next time an object is allocated or finished in the
obstack. If you are not growing an object, you can make the new
alignment mask take effect immediately by calling obstack_finish
.
This will finish a zero-length object and then do proper alignment for
the next object.
Obstacks work by allocating space for themselves in large chunks, and then parceling out space in the chunks to satisfy your requests. Chunks are normally 4096 bytes long unless you specify a different chunk size. The chunk size includes 8 bytes of overhead that are not actually used for storing objects. Regardless of the specified size, longer chunks will be allocated when necessary for long objects.
The obstack library allocates chunks by calling the function
obstack_chunk_alloc
, which you must define. When a chunk is no
longer needed because you have freed all the objects in it, the obstack
library frees the chunk by calling obstack_chunk_free
, which you
must also define.
These two must be defined (as macros) or declared (as functions) in each
source file that uses obstack_init
(see Creating Obstacks).
Most often they are defined as macros like this:
#define obstack_chunk_alloc malloc #define obstack_chunk_free free
Note that these are simple macros (no arguments). Macro definitions with
arguments will not work! It is necessary that obstack_chunk_alloc
or obstack_chunk_free
, alone, expand into a function name if it is
not itself a function name.
If you allocate chunks with malloc
, the chunk size should be a
power of 2. The default chunk size, 4096, was chosen because it is long
enough to satisfy many typical requests on the obstack yet short enough
not to waste too much memory in the portion of the last chunk not yet used.
int obstack_chunk_size (struct obstack *obstack-ptr) | Macro |
This returns the chunk size of the given obstack. |
Since this macro expands to an lvalue, you can specify a new chunk size by
assigning it a new value. Doing so does not affect the chunks already
allocated, but will change the size of chunks allocated for that particular
obstack in the future. It is unlikely to be useful to make the chunk size
smaller, but making it larger might improve efficiency if you are
allocating many objects whose size is comparable to the chunk size. Here
is how to do so cleanly:
if (obstack_chunk_size (obstack_ptr) < new-chunk-size) obstack_chunk_size (obstack_ptr) = new-chunk-size;
Here is a summary of all the functions associated with obstacks. Each
takes the address of an obstack (struct obstack *
) as its first
argument.
void obstack_init (struct obstack *obstack-ptr)
void *obstack_alloc (struct obstack *obstack-ptr, int size)
void *obstack_copy (struct obstack *obstack-ptr, void *address, int size)
void *obstack_copy0 (struct obstack *obstack-ptr, void *address, int size)
void obstack_free (struct obstack *obstack-ptr, void *object)
void obstack_blank (struct obstack *obstack-ptr, int size)
void obstack_grow (struct obstack *obstack-ptr, void *address, int size)
void obstack_grow0 (struct obstack *obstack-ptr, void *address, int size)
void obstack_1grow (struct obstack *obstack-ptr, char data-char)
void *obstack_finish (struct obstack *obstack-ptr)
int obstack_object_size (struct obstack *obstack-ptr)
void obstack_blank_fast (struct obstack *obstack-ptr, int size)
void obstack_1grow_fast (struct obstack *obstack-ptr, char data-char)
int obstack_room (struct obstack *obstack-ptr)
int obstack_alignment_mask (struct obstack *obstack-ptr)
int obstack_chunk_size (struct obstack *obstack-ptr)
void *obstack_base (struct obstack *obstack-ptr)
void *obstack_next_free (struct obstack *obstack-ptr)
The function alloca
supports a kind of half-dynamic allocation in
which blocks are allocated dynamically but freed automatically.
Allocating a block with alloca
is an explicit action; you can
allocate as many blocks as you wish, and compute the size at run time. But
all the blocks are freed when you exit the function that alloca
was
called from, just as if they were automatic variables declared in that
function. There is no way to free the space explicitly.
The prototype for alloca
is in stdlib.h
. This function is
a BSD extension.
void * alloca (size_t size); | Function |
The return value of alloca is the address of a block of size
bytes of memory, allocated in the stack frame of the calling function.
|
Do not use alloca
inside the arguments of a function call--you
will get unpredictable results, because the stack space for the
alloca
would appear on the stack in the middle of the space for
the function arguments. An example of what to avoid is foo (x,
alloca (4), y)
.
alloca
.
alloca
.
alloca
.
alloca
ExampleAs an example of the use of alloca
, here is a function that opens
a file name made from concatenating two argument strings, and returns a
file descriptor or minus one signifying failure:
int open2 (char *str1, char *str2, int flags, int mode) { char *name = (char *) alloca (strlen (str1) + strlen (str2) + 1); stpcpy (stpcpy (name, str1), str2); return open (name, flags, mode); }
Here is how you would get the same results with malloc
and
free
:
int open2 (char *str1, char *str2, int flags, int mode) { char *name = (char *) malloc (strlen (str1) + strlen (str2) + 1); int desc; if (name == 0) fatal ("virtual memory exceeded"); stpcpy (stpcpy (name, str1), str2); desc = open (name, flags, mode); free (name); return desc; }
As you can see, it is simpler with alloca
. But alloca
has
other, more important advantages, and some disadvantages.
alloca
Here are the reasons why alloca
may be preferable to malloc
:
alloca
wastes very little space and is very fast. (It is
open-coded by the GNU C compiler.)
alloca
does not have separate pools for different sizes of
block, space used for any size block can be reused for any other size.
alloca
does not cause memory fragmentation.
longjmp
(see Non-Local Exits)
automatically free the space allocated with alloca
when they exit
through the function that called alloca
. This is the most
important reason to use alloca
.
To illustrate this, suppose you have a function
open_or_report_error
which returns a descriptor, like
open
, if it succeeds, but does not return to its caller if it
fails. If the file cannot be opened, it prints an error message and
jumps out to the command level of your program using longjmp
.
Let's change open2
(see Alloca Example) to use this
subroutine:
int open2 (char *str1, char *str2, int flags, int mode) { char *name = (char *) alloca (strlen (str1) + strlen (str2) + 1); stpcpy (stpcpy (name, str1), str2); return open_or_report_error (name, flags, mode); }
Because of the way alloca
works, the memory it allocates is
freed even when an error occurs, with no special effort required.
By contrast, the previous definition of open2
(which uses
malloc
and free
) would develop a memory leak if it were
changed in this way. Even if you are willing to make more changes to
fix it, there is no easy way to do so.
alloca
These are the disadvantages of alloca
in comparison with
malloc
:
alloca
, so it is less
portable. However, a slower emulation of alloca
written in C
is available for use on systems with this deficiency.
In GNU C, you can replace most uses of alloca
with an array of
variable size. Here is how open2
would look then:
int open2 (char *str1, char *str2, int flags, int mode) { char name[strlen (str1) + strlen (str2) + 1]; stpcpy (stpcpy (name, str1), str2); return open (name, flags, mode); }
But alloca
is not always equivalent to a variable-sized array, for
several reasons:
alloca
remains until the end of the function.
alloca
within a loop, allocating an
additional block on each iteration. This is impossible with
variable-sized arrays.
Note: If you mix use of alloca
and variable-sized arrays
within one function, exiting a scope in which a variable-sized array was
declared frees all blocks allocated with alloca
during the
execution of that scope.
The symbols in this section are declared in unistd.h
.
You will not normally use the functions in this section, because the functions described in Memory Allocation are easier to use. Those are interfaces to a GNU C Library memory allocator that uses the functions below itself. The functions below are simple interfaces to system calls.
int brk (void *addr) | Function |
The address of the end of a segment is defined to be the address of the last byte in the segment plus 1. The function has no effect if addr is lower than the low end of the data segment. (This is considered success, by the way). The function fails if it would cause the data segment to overlap another segment or exceed the process' data storage limit (see Limits on Resources). The function is named for a common historical case where data storage and the stack are in the same segment. Data storage allocation grows upward from the bottom of the segment while the stack grows downward toward it from the top of the segment and the curtain between them is called the break. The return value is zero on success. On failure, the return value is
|
int sbrk (ptrdiff_t delta) | Function |
This function is the same as brk except that you specify the new
end of the data segment as an offset delta from the current end
and on success the return value is the address of the resulting end of
the data segment instead of zero.
This means you can use |
You can tell the system to associate a particular virtual memory page with a real page frame and keep it that way -- i.e. cause the page to be paged in if it isn't already and mark it so it will never be paged out and consequently will never cause a page fault. This is called locking a page.
The functions in this chapter lock and unlock the calling process' pages.
Because page faults cause paged out pages to be paged in transparently, a process rarely needs to be concerned about locking pages. However, there are two reasons people sometimes are:
A process that needs to lock pages for this reason probably also needs priority among other processes for use of the CPU. See Priority.
In some cases, the programmer knows better than the system's demand paging allocator which pages should remain in real memory to optimize system performance. In this case, locking pages can help.
Be aware that when you lock a page, that's one fewer page frame that can be used to back other virtual memory (by the same or other processes), which can mean more page faults, which means the system runs more slowly. In fact, if you lock enough memory, some programs may not be able to run at all for lack of real memory.
A memory lock is associated with a virtual page, not a real frame. The paging rule is: If a frame backs at least one locked page, don't page it out.
Memory locks do not stack. I.e. you can't lock a particular page twice so that it has to be unlocked twice before it is truly unlocked. It is either locked or it isn't.
A memory lock persists until the process that owns the memory explicitly unlocks it. (But process termination and exec cause the virtual memory to cease to exist, which you might say means it isn't locked any more).
Memory locks are not inherited by child processes. (But note that on a modern Unix system, immediately after a fork, the parent's and the child's virtual address space are backed by the same real page frames, so the child enjoys the parent's locks). See Creating a Process.
Because of its ability to impact other processes, only the superuser can lock a page. Any process can unlock its own page.
The system sets limits on the amount of memory a process can have locked and the amount of real memory it can have dedicated to it. See Limits on Resources.
In Linux, locked pages aren't as locked as you might think. Two virtual pages that are not shared memory can nonetheless be backed by the same real frame. The kernel does this in the name of efficiency when it knows both virtual pages contain identical data, and does it even if one or both of the virtual pages are locked.
But when a process modifies one of those pages, the kernel must get it a separate frame and fill it with the page's data. This is known as a copy-on-write page fault. It takes a small amount of time and in a pathological case, getting that frame may require I/O.
To make sure this doesn't happen to your program, don't just lock the pages. Write to them as well, unless you know you won't write to them ever. And to make sure you have pre-allocated frames for your stack, enter a scope that declares a C automatic variable larger than the maximum stack size you will need, set it to something, then return from its scope.
The symbols in this section are declared in sys/mman.h
. These
functions are defined by POSIX.1b, but their availability depends on
your kernel. If your kernel doesn't allow these functions, they exist
but always fail. They are available with a Linux kernel.
Portability Note: POSIX.1b requires that when the mlock
and munlock
functions are available, the file unistd.h
define the macro _POSIX_MEMLOCK_RANGE
and the file
limits.h
define the macro PAGESIZE
to be the size of a
memory page in bytes. It requires that when the mlockall
and
munlockall
functions are available, the unistd.h
file
define the macro _POSIX_MEMLOCK
. The GNU C library conforms to
this requirement.
int mlock (const void *addr, size_t len) | Function |
The range of memory starts at address addr and is len bytes long. Actually, since you must lock whole pages, it is the range of pages that include any part of the specified range. When the function returns successfully, each of those pages is backed by (connected to) a real frame (is resident) and is marked to stay that way. This means the function may cause page-ins and have to wait for them. When the function fails, it does not affect the lock status of any pages. The return value is zero if the function succeeds. Otherwise, it is
You can lock all a process' memory with To avoid all page faults in a C program, you have to use
|
int munlock (const void *addr, size_t len) | Function |
|
int mlockall (int flags) | Function |
flags is a string of single bit flags represented by the following
macros. They tell
When the function returns successfully, and you specified
When the process is in When the function fails, it does not affect the lock status of any pages or the future locking mode. The return value is zero if the function succeeds. Otherwise, it is
You can lock just specific pages with |
int munlockall (void) | Function |
The return value is zero if the function succeeds. Otherwise, it is
|
Programs that work with characters and strings often need to classify a
character--is it alphabetic, is it a digit, is it whitespace, and so
on--and perform case conversion operations on characters. The
functions in the header file ctype.h
are provided for this
purpose.
Since the choice of locale and character set can alter the
classifications of particular character codes, all of these functions
are affected by the current locale. (More precisely, they are affected
by the locale currently selected for character classification--the
LC_CTYPE
category; see Locale Categories.)
The ISO C standard specifies two different sets of functions. The
one set works on char
type characters, the other one on
wchar_t
wide characters (see Extended Char Intro).
This section explains the library functions for classifying characters.
For example, isalpha
is the function to test for an alphabetic
character. It takes one argument, the character to test, and returns a
nonzero integer if the character is alphabetic, and zero otherwise. You
would use it like this:
if (isalpha (c)) printf ("The character `%c' is alphabetic.\n", c);
Each of the functions in this section tests for membership in a
particular class of characters; each has a name starting with is
.
Each of them takes one argument, which is a character to test, and
returns an int
which is treated as a boolean value. The
character argument is passed as an int
, and it may be the
constant value EOF
instead of a real character.
The attributes of any given character can vary between locales. See Locales, for more information on locales.
These functions are declared in the header file ctype.h
.
int islower (int c) | Function |
Returns true if c is a lower-case letter. The letter need not be from the Latin alphabet, any alphabet representable is valid. |
int isupper (int c) | Function |
Returns true if c is an upper-case letter. The letter need not be from the Latin alphabet, any alphabet representable is valid. |
int isalpha (int c) | Function |
Returns true if c is an alphabetic character (a letter). If
islower or isupper is true of a character, then
isalpha is also true.
In some locales, there may be additional characters for which
|
int isdigit (int c) | Function |
Returns true if c is a decimal digit (0 through 9 ).
|
int isalnum (int c) | Function |
Returns true if c is an alphanumeric character (a letter or
number); in other words, if either isalpha or isdigit is
true of a character, then isalnum is also true.
|
int isxdigit (int c) | Function |
Returns true if c is a hexadecimal digit.
Hexadecimal digits include the normal decimal digits 0 through
9 and the letters A through F and
a through f .
|
int ispunct (int c) | Function |
Returns true if c is a punctuation character. This means any printing character that is not alphanumeric or a space character. |
int isspace (int c) | Function |
Returns true if c is a whitespace character. In the standard
"C" locale, isspace returns true for only the standard
whitespace characters:
|
int isblank (int c) | Function |
Returns true if c is a blank character; that is, a space or a tab. This function is a GNU extension. |
int isgraph (int c) | Function |
Returns true if c is a graphic character; that is, a character that has a glyph associated with it. The whitespace characters are not considered graphic. |
int isprint (int c) | Function |
Returns true if c is a printing character. Printing characters
include all the graphic characters, plus the space ( ) character.
|
int iscntrl (int c) | Function |
Returns true if c is a control character (that is, a character that is not a printing character). |
int isascii (int c) | Function |
Returns true if c is a 7-bit unsigned char value that fits
into the US/UK ASCII character set. This function is a BSD extension
and is also an SVID extension.
|
This section explains the library functions for performing conversions
such as case mappings on characters. For example, toupper
converts any character to upper case if possible. If the character
can't be converted, toupper
returns it unchanged.
These functions take one argument of type int
, which is the
character to convert, and return the converted character as an
int
. If the conversion is not applicable to the argument given,
the argument is returned unchanged.
Compatibility Note: In pre-ISO C dialects, instead of
returning the argument unchanged, these functions may fail when the
argument is not suitable for the conversion. Thus for portability, you
may need to write islower(c) ? toupper(c) : c
rather than just
toupper(c)
.
These functions are declared in the header file ctype.h
.
int tolower (int c) | Function |
If c is an upper-case letter, tolower returns the corresponding
lower-case letter. If c is not an upper-case letter,
c is returned unchanged.
|
int toupper (int c) | Function |
If c is a lower-case letter, toupper returns the corresponding
upper-case letter. Otherwise c is returned unchanged.
|
int toascii (int c) | Function |
This function converts c to a 7-bit unsigned char value
that fits into the US/UK ASCII character set, by clearing the high-order
bits. This function is a BSD extension and is also an SVID extension.
|
int _tolower (int c) | Function |
This is identical to tolower , and is provided for compatibility
with the SVID. See SVID.
|
int _toupper (int c) | Function |
This is identical to toupper , and is provided for compatibility
with the SVID.
|
Amendment 1 to ISO C90 defines functions to classify wide
characters. Although the original ISO C90 standard already defined
the type wchar_t
, no functions operating on them were defined.
The general design of the classification functions for wide characters
is more general. It allows extensions to the set of available
classifications, beyond those which are always available. The POSIX
standard specifies how extensions can be made, and this is already
implemented in the GNU C library implementation of the localedef
program.
The character class functions are normally implemented with bitsets, with a bitset per character. For a given character, the appropriate bitset is read from a table and a test is performed as to whether a certain bit is set. Which bit is tested for is determined by the class.
For the wide character classification functions this is made visible.
There is a type classification type defined, a function to retrieve this
value for a given class, and a function to test whether a given
character is in this class, using the classification value. On top of
this the normal character classification functions as used for
char
objects can be defined.
wctype_t | Data type |
The wctype_t can hold a value which represents a character class.
The only defined way to generate such a value is by using the
wctype function.
This type is defined in |
wctype_t wctype (const char *property) | Function |
The wctype returns a value representing a class of wide
characters which is identified by the string property. Beside
some standard properties each locale can define its own ones. In case
no property with the given name is known for the current locale
selected for the LC_CTYPE category, the function returns zero.
The properties known in every locale are:
This function is declared in |
To test the membership of a character to one of the non-standard classes the ISO C standard defines a completely new function.
int iswctype (wint_t wc, wctype_t desc) | Function |
This function returns a nonzero value if wc is in the character
class specified by desc. desc must previously be returned
by a successful call to wctype .
This function is declared in |
To make it easier to use the commonly-used classification functions,
they are defined in the C library. There is no need to use
wctype
if the property string is one of the known character
classes. In some situations it is desirable to construct the property
strings, and then it is important that wctype
can also handle the
standard classes.
int iswalnum (wint_t wc) | Function |
This function returns a nonzero value if wc is an alphanumeric
character (a letter or number); in other words, if either iswalpha
or iswdigit is true of a character, then iswalnum is also
true.
This function can be implemented using
iswctype (wc, wctype ("alnum")) It is declared in |
int iswalpha (wint_t wc) | Function |
Returns true if wc is an alphabetic character (a letter). If
iswlower or iswupper is true of a character, then
iswalpha is also true.
In some locales, there may be additional characters for which
This function can be implemented using
iswctype (wc, wctype ("alpha")) It is declared in |
int iswcntrl (wint_t wc) | Function |
Returns true if wc is a control character (that is, a character that
is not a printing character).
This function can be implemented using
iswctype (wc, wctype ("cntrl")) It is declared in |
int iswdigit (wint_t wc) | Function |
Returns true if wc is a digit (e.g., 0 through 9 ).
Please note that this function does not only return a nonzero value for
decimal digits, but for all kinds of digits. A consequence is
that code like the following will not work unconditionally for
wide characters:
n = 0; while (iswdigit (*wc)) { n *= 10; n += *wc++ - L'0'; } This function can be implemented using
iswctype (wc, wctype ("digit")) It is declared in |
int iswgraph (wint_t wc) | Function |
Returns true if wc is a graphic character; that is, a character
that has a glyph associated with it. The whitespace characters are not
considered graphic.
This function can be implemented using
iswctype (wc, wctype ("graph")) It is declared in |
int iswlower (wint_t wc) | Function |
Returns true if wc is a lower-case letter. The letter need not be
from the Latin alphabet, any alphabet representable is valid.
This function can be implemented using
iswctype (wc, wctype ("lower")) It is declared in |
int iswprint (wint_t wc) | Function |
Returns true if wc is a printing character. Printing characters
include all the graphic characters, plus the space ( ) character.
This function can be implemented using
iswctype (wc, wctype ("print")) It is declared in |
int iswpunct (wint_t wc) | Function |
Returns true if wc is a punctuation character.
This means any printing character that is not alphanumeric or a space
character.
This function can be implemented using
iswctype (wc, wctype ("punct")) It is declared in |
int iswspace (wint_t wc) | Function |
Returns true if wc is a whitespace character. In the standard
"C" locale, iswspace returns true for only the standard
whitespace characters:
This function can be implemented using
iswctype (wc, wctype ("space")) It is declared in |
int iswupper (wint_t wc) | Function |
Returns true if wc is an upper-case letter. The letter need not be
from the Latin alphabet, any alphabet representable is valid.
This function can be implemented using
iswctype (wc, wctype ("upper")) It is declared in |
int iswxdigit (wint_t wc) | Function |
Returns true if wc is a hexadecimal digit.
Hexadecimal digits include the normal decimal digits 0 through
9 and the letters A through F and
a through f .
This function can be implemented using
iswctype (wc, wctype ("xdigit")) It is declared in |
The GNU C library also provides a function which is not defined in the ISO C standard but which is available as a version for single byte characters as well.
int iswblank (wint_t wc) | Function |
Returns true if wc is a blank character; that is, a space or a tab.
This function is a GNU extension. It is declared in wchar.h .
|
The first note is probably not astonishing but still occasionally a
cause of problems. The iswXXX
functions can be implemented
using macros and in fact, the GNU C library does this. They are still
available as real functions but when the wctype.h
header is
included the macros will be used. This is the same as the
char
type versions of these functions.
The second note covers something new. It can be best illustrated by a
(real-world) example. The first piece of code is an excerpt from the
original code. It is truncated a bit but the intention should be clear.
int is_in_class (int c, const char *class) { if (strcmp (class, "alnum") == 0) return isalnum (c); if (strcmp (class, "alpha") == 0) return isalpha (c); if (strcmp (class, "cntrl") == 0) return iscntrl (c); ... return 0; }
Now, with the wctype
and iswctype
you can avoid the
if
cascades, but rewriting the code as follows is wrong:
int is_in_class (int c, const char *class) { wctype_t desc = wctype (class); return desc ? iswctype ((wint_t) c, desc) : 0; }
The problem is that it is not guaranteed that the wide character
representation of a single-byte character can be found using casting.
In fact, usually this fails miserably. The correct solution to this
problem is to write the code as follows:
int is_in_class (int c, const char *class) { wctype_t desc = wctype (class); return desc ? iswctype (btowc (c), desc) : 0; }
See Converting a Character, for more information on btowc
.
Note that this change probably does not improve the performance
of the program a lot since the wctype
function still has to make
the string comparisons. It gets really interesting if the
is_in_class
function is called more than once for the
same class name. In this case the variable desc could be computed
once and reused for all the calls. Therefore the above form of the
function is probably not the final one.
The classification functions are also generalized by the ISO C
standard. Instead of just allowing the two standard mappings, a
locale can contain others. Again, the localedef
program
already supports generating such locale data files.
wctrans_t | Data Type |
This data type is defined as a scalar type which can hold a value
representing the locale-dependent character mapping. There is no way to
construct such a value apart from using the return value of the
wctrans function.
This type is defined in |
wctrans_t wctrans (const char *property) | Function |
The wctrans function has to be used to find out whether a named
mapping is defined in the current locale selected for the
LC_CTYPE category. If the returned value is non-zero, you can use
it afterwards in calls to towctrans . If the return value is
zero no such mapping is known in the current locale.
Beside locale-specific mappings there are two mappings which are guaranteed to be available in every locale:
These functions are declared in |
wint_t towctrans (wint_t wc, wctrans_t desc) | Function |
towctrans maps the input character wc
according to the rules of the mapping for which desc is a
descriptor, and returns the value it finds. desc must be
obtained by a successful call to wctrans .
This function is declared in |
For the generally available mappings, the ISO C standard defines
convenient shortcuts so that it is not necessary to call wctrans
for them.
wint_t towlower (wint_t wc) | Function |
If wc is an upper-case letter, towlower returns the corresponding
lower-case letter. If wc is not an upper-case letter,
wc is returned unchanged.
towctrans (wc, wctrans ("tolower")) This function is declared in |
wint_t towupper (wint_t wc) | Function |
If wc is a lower-case letter, towupper returns the corresponding
upper-case letter. Otherwise wc is returned unchanged.
towctrans (wc, wctrans ("toupper")) This function is declared in |
The same warnings given in the last section for the use of the wide
character classification functions apply here. It is not possible to
simply cast a char
type value to a wint_t
and use it as an
argument to towctrans
calls.
Operations on strings (or arrays of characters) are an important part of
many programs. The GNU C library provides an extensive set of string
utility functions, including functions for copying, concatenating,
comparing, and searching strings. Many of these functions can also
operate on arbitrary regions of storage; for example, the memcpy
function can be used to copy the contents of any kind of array.
It's fairly common for beginning C programmers to "reinvent the wheel" by duplicating this functionality in their own code, but it pays to become familiar with the library functions and to make use of them, since this offers benefits in maintenance, efficiency, and portability.
For instance, you could easily compare one string to another in two
lines of C code, but if you use the built-in strcmp
function,
you're less likely to make a mistake. And, since these library
functions are typically highly optimized, your program may run faster
too.
This section is a quick summary of string concepts for beginning C programmers. It describes how character strings are represented in C and some common pitfalls. If you are already familiar with this material, you can skip this section.
A string is an array of char
objects. But string-valued
variables are usually declared to be pointers of type char *
.
Such variables do not include space for the text of a string; that has
to be stored somewhere else--in an array variable, a string constant,
or dynamically allocated memory (see Memory Allocation). It's up to
you to store the address of the chosen memory space into the pointer
variable. Alternatively you can store a null pointer in the
pointer variable. The null pointer does not point anywhere, so
attempting to reference the string it points to gets an error.
"string" normally refers to multibyte character strings as opposed to
wide character strings. Wide character strings are arrays of type
wchar_t
and as for multibyte character strings usually pointers
of type wchar_t *
are used.
By convention, a null character, '\0'
, marks the end of a
multibyte character string and the null wide character,
L'\0'
, marks the end of a wide character string. For example, in
testing to see whether the char *
variable p points to a
null character marking the end of a string, you can write
!*p
or *p == '\0'
.
A null character is quite different conceptually from a null pointer,
although both are represented by the integer 0
.
String literals appear in C program source as strings of
characters between double-quote characters ("
) where the initial
double-quote character is immediately preceded by a capital L
(ell) character (as in L"foo"
). In ISO C, string literals
can also be formed by string concatenation: "a" "b"
is the
same as "ab"
. For wide character strings one can either use
L"a" L"b"
or L"a" "b"
. Modification of string literals is
not allowed by the GNU C compiler, because literals are placed in
read-only storage.
Character arrays that are declared const
cannot be modified
either. It's generally good style to declare non-modifiable string
pointers to be of type const char *
, since this often allows the
C compiler to detect accidental modifications as well as providing some
amount of documentation about what your program intends to do with the
string.
The amount of memory allocated for the character array may extend past the null character that normally marks the end of the string. In this document, the term allocated size is always used to refer to the total amount of memory allocated for the string, while the term length refers to the number of characters up to (but not including) the terminating null character.
A notorious source of program bugs is trying to put more characters in a string than fit in its allocated size. When writing code that extends strings or moves characters into a pre-allocated array, you should be very careful to keep track of the length of the text and make explicit checks for overflowing the array. Many of the library functions do not do this for you! Remember also that you need to allocate an extra byte to hold the null character that marks the end of the string.
Originally strings were sequences of bytes where each byte represents a single character. This is still true today if the strings are encoded using a single-byte character encoding. Things are different if the strings are encoded using a multibyte encoding (for more information on encodings see Extended Char Intro). There is no difference in the programming interface for these two kind of strings; the programmer has to be aware of this and interpret the byte sequences accordingly.
But since there is no separate interface taking care of these
differences the byte-based string functions are sometimes hard to use.
Since the count parameters of these functions specify bytes a call to
strncpy
could cut a multibyte character in the middle and put an
incomplete (and therefore unusable) byte sequence in the target buffer.
To avoid these problems later versions of the ISO C standard introduce a second set of functions which are operating on wide characters (see Extended Char Intro). These functions don't have the problems the single-byte versions have since every wide character is a legal, interpretable value. This does not mean that cutting wide character strings at arbitrary points is without problems. It normally is for alphabet-based languages (except for non-normalized text) but languages based on syllables still have the problem that more than one wide character is necessary to complete a logical unit. This is a higher level problem which the C library functions are not designed to solve. But it is at least good that no invalid byte sequences can be created. Also, the higher level functions can also much easier operate on wide character than on multibyte characters so that a general advise is to use wide characters internally whenever text is more than simply copied.
The remaining of this chapter will discuss the functions for handling wide character strings in parallel with the discussion of the multibyte character strings since there is almost always an exact equivalent available.
This chapter describes both functions that work on arbitrary arrays or blocks of memory, and functions that are specific to null-terminated arrays of characters and wide characters.
Functions that operate on arbitrary blocks of memory have names
beginning with mem
and wmem
(such as memcpy
and
wmemcpy
) and invariably take an argument which specifies the size
(in bytes and wide characters respectively) of the block of memory to
operate on. The array arguments and return values for these functions
have type void *
or wchar_t
. As a matter of style, the
elements of the arrays used with the mem
functions are referred
to as "bytes". You can pass any kind of pointer to these functions,
and the sizeof
operator is useful in computing the value for the
size argument. Parameters to the wmem
functions must be of type
wchar_t *
. These functions are not really usable with anything
but arrays of this type.
In contrast, functions that operate specifically on strings and wide
character strings have names beginning with str
and wcs
respectively (such as strcpy
and wcscpy
) and look for a
null character to terminate the string instead of requiring an explicit
size argument to be passed. (Some of these functions accept a specified
maximum length, but they also check for premature termination with a
null character.) The array arguments and return values for these
functions have type char *
and wchar_t *
respectively, and
the array elements are referred to as "characters" and "wide
characters".
In many cases, there are both mem
and str
/wcs
versions of a function. The one that is more appropriate to use depends
on the exact situation. When your program is manipulating arbitrary
arrays or blocks of storage, then you should always use the mem
functions. On the other hand, when you are manipulating null-terminated
strings it is usually more convenient to use the str
/wcs
functions, unless you already know the length of the string in advance.
The wmem
functions should be used for wide character arrays with
known size.
Some of the memory and string functions take single characters as
arguments. Since a value of type char
is automatically promoted
into an value of type int
when used as a parameter, the functions
are declared with int
as the type of the parameter in question.
In case of the wide character function the situation is similarly: the
parameter type for a single wide character is wint_t
and not
wchar_t
. This would for many implementations not be necessary
since the wchar_t
is large enough to not be automatically
promoted, but since the ISO C standard does not require such a
choice of types the wint_t
type is used.
You can get the length of a string using the strlen
function.
This function is declared in the header file string.h
.
size_t strlen (const char *s) | Function |
The strlen function returns the length of the null-terminated
string s in bytes. (In other words, it returns the offset of the
terminating null character within the array.)
For example,
strlen ("hello, world") => 12 When applied to a character array, the char string[32] = "hello, world"; sizeof (string) => 32 strlen (string) => 12 But beware, this will not work unless string is the character
array itself, not a pointer to it. For example:
char string[32] = "hello, world"; char *ptr = string; sizeof (string) => 32 sizeof (ptr) => 4 /* (on a machine with 4 byte pointers) */ This is an easy mistake to make when you are working with functions that take string arguments; those arguments are always pointers, not arrays. It must also be noted that for multibyte encoded strings the return
value does not have to correspond to the number of characters in the
string. To get this value the string can be converted to wide
characters and /* The input is in This is cumbersome to do so if the number of characters (as opposed to bytes) is needed often it is better to work with wide characters. |
The wide character equivalent is declared in wchar.h
.
size_t wcslen (const wchar_t *ws) | Function |
The wcslen function is the wide character equivalent to
strlen . The return value is the number of wide characters in the
wide character string pointed to by ws (this is also the offset of
the terminating null wide character of ws).
Since there are no multi wide character sequences making up one character the return value is not only the offset in the array, it is also the number of wide characters. This function was introduced in Amendment 1 to ISO C90. |
size_t strnlen (const char *s, size_t maxlen) | Function |
The strnlen function returns the length of the string s in
bytes if this length is smaller than maxlen bytes. Otherwise it
returns maxlen. Therefore this function is equivalent to
(strlen (s) < n ? strlen (s) : maxlen) but it
is more efficient and works even if the string s is not
null-terminated.
char string[32] = "hello, world"; strnlen (string, 32) => 12 strnlen (string, 5) => 5 This function is a GNU extension and is declared in |
size_t wcsnlen (const wchar_t *ws, size_t maxlen) | Function |
wcsnlen is the wide character equivalent to strnlen . The
maxlen parameter specifies the maximum number of wide characters.
This function is a GNU extension and is declared in |
You can use the functions described in this section to copy the contents
of strings and arrays, or to append the contents of one string to
another. The str
and mem
functions are declared in the
header file string.h
while the wstr
and wmem
functions are declared in the file wchar.h
.
A helpful way to remember the ordering of the arguments to the functions in this section is that it corresponds to an assignment expression, with the destination array specified to the left of the source array. All of these functions return the address of the destination array.
Most of these functions do not work properly if the source and destination arrays overlap. For example, if the beginning of the destination array overlaps the end of the source array, the original contents of that part of the source array may get overwritten before it is copied. Even worse, in the case of the string functions, the null character marking the end of the string may be lost, and the copy function might get stuck in a loop trashing all the memory allocated to your program.
All functions that have problems copying between overlapping arrays are
explicitly identified in this manual. In addition to functions in this
section, there are a few others like sprintf
(see Formatted Output Functions) and scanf
(see Formatted Input Functions).
void * memcpy (void *restrict to, const void *restrict from, size_t size) | Function |
The memcpy function copies size bytes from the object
beginning at from into the object beginning at to. The
behavior of this function is undefined if the two arrays to and
from overlap; use memmove instead if overlapping is possible.
The value returned by Here is an example of how you might use struct foo *oldarray, *newarray; int arraysize; ... memcpy (new, old, arraysize * sizeof (struct foo)); |
wchar_t * wmemcpy (wchar_t *restrict wto, const wchar_t *restruct wfrom, size_t size) | Function |
The wmemcpy function copies size wide characters from the object
beginning at wfrom into the object beginning at wto. The
behavior of this function is undefined if the two arrays wto and
wfrom overlap; use wmemmove instead if overlapping is possible.
The following is a possible implementation of wchar_t * wmemcpy (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size) { return (wchar_t *) memcpy (wto, wfrom, size * sizeof (wchar_t)); } The value returned by This function was introduced in Amendment 1 to ISO C90. |
void * mempcpy (void *restrict to, const void *restrict from, size_t size) | Function |
The mempcpy function is nearly identical to the memcpy
function. It copies size bytes from the object beginning at
from into the object pointed to by to. But instead of
returning the value of to it returns a pointer to the byte
following the last written byte in the object beginning at to.
I.e., the value is ((void *) ((char *) to + size)) .
This function is useful in situations where a number of objects shall be
copied to consecutive memory positions.
void * combine (void *o1, size_t s1, void *o2, size_t s2) { void *result = malloc (s1 + s2); if (result != NULL) mempcpy (mempcpy (result, o1, s1), o2, s2); return result; } This function is a GNU extension. |
wchar_t * wmempcpy (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size) | Function |
The wmempcpy function is nearly identical to the wmemcpy
function. It copies size wide characters from the object
beginning at wfrom into the object pointed to by wto. But
instead of returning the value of wto it returns a pointer to the
wide character following the last written wide character in the object
beginning at wto. I.e., the value is wto + size .
This function is useful in situations where a number of objects shall be copied to consecutive memory positions. The following is a possible implementation of wchar_t * wmempcpy (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size) { return (wchar_t *) mempcpy (wto, wfrom, size * sizeof (wchar_t)); } This function is a GNU extension. |
void * memmove (void *to, const void *from, size_t size) | Function |
memmove copies the size bytes at from into the
size bytes at to, even if those two blocks of space
overlap. In the case of overlap, memmove is careful to copy the
original values of the bytes in the block at from, including those
bytes which also belong to the block at to.
The value returned by |
wchar_t * wmemmove (wchar *wto, const wchar_t *wfrom, size_t size) | Function |
wmemmove copies the size wide characters at wfrom
into the size wide characters at wto, even if those two
blocks of space overlap. In the case of overlap, memmove is
careful to copy the original values of the wide characters in the block
at wfrom, including those wide characters which also belong to the
block at wto.
The following is a possible implementation of wchar_t * wmempcpy (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size) { return (wchar_t *) mempcpy (wto, wfrom, size * sizeof (wchar_t)); } The value returned by This function is a GNU extension. |
void * memccpy (void *restrict to, const void *restrict from, int c, size_t size) | Function |
This function copies no more than size bytes from from to to, stopping if a byte matching c is found. The return value is a pointer into to one byte past where c was copied, or a null pointer if no byte matching c appeared in the first size bytes of from. |
void * memset (void *block, int c, size_t size) | Function |
This function copies the value of c (converted to an
unsigned char ) into each of the first size bytes of the
object beginning at block. It returns the value of block.
|
wchar_t * wmemset (wchar_t *block, wchar_t wc, size_t size) | Function |
This function copies the value of wc into each of the first size wide characters of the object beginning at block. It returns the value of block. |
char * strcpy (char *restrict to, const char *restrict from) | Function |
This copies characters from the string from (up to and including
the terminating null character) into the string to. Like
memcpy , this function has undefined results if the strings
overlap. The return value is the value of to.
|
wchar_t * wcscpy (wchar_t *restrict wto, const wchar_t *restrict wfrom) | Function |
This copies wide characters from the string wfrom (up to and
including the terminating null wide character) into the string
wto. Like wmemcpy , this function has undefined results if
the strings overlap. The return value is the value of wto.
|
char * strncpy (char *restrict to, const char *restrict from, size_t size) | Function |
This function is similar to strcpy but always copies exactly
size characters into to.
If the length of from is more than size, then If the length of from is less than size, then The behavior of Using |
wchar_t * wcsncpy (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size) | Function |
This function is similar to wcscpy but always copies exactly
size wide characters into wto.
If the length of wfrom is more than size, then
If the length of wfrom is less than size, then
The behavior of Using |
char * strdup (const char *s) | Function |
This function copies the null-terminated string s into a newly
allocated string. The string is allocated using malloc ; see
Unconstrained Allocation. If malloc cannot allocate space
for the new string, strdup returns a null pointer. Otherwise it
returns a pointer to the new string.
|
wchar_t * wcsdup (const wchar_t *ws) | Function |
This function copies the null-terminated wide character string ws
into a newly allocated string. The string is allocated using
malloc ; see Unconstrained Allocation. If malloc
cannot allocate space for the new string, wcsdup returns a null
pointer. Otherwise it returns a pointer to the new wide character
string.
This function is a GNU extension. |
char * strndup (const char *s, size_t size) | Function |
This function is similar to strdup but always copies at most
size characters into the newly allocated string.
If the length of s is more than size, then This function is different to
|
char * stpcpy (char *restrict to, const char *restrict from) | Function |
This function is like strcpy , except that it returns a pointer to
the end of the string to (that is, the address of the terminating
null character to + strlen (from) ) rather than the beginning.
For example, this program uses #include <string.h> #include <stdio.h> int main (void) { char buffer[10]; char *to = buffer; to = stpcpy (to, "foo"); to = stpcpy (to, "bar"); puts (buffer); return 0; } This function is not part of the ISO or POSIX standards, and is not customary on Unix systems, but we did not invent it either. Perhaps it comes from MS-DOG. Its behavior is undefined if the strings overlap. The function is
declared in |
wchar_t * wcpcpy (wchar_t *restrict wto, const wchar_t *restrict wfrom) | Function |
This function is like wcscpy , except that it returns a pointer to
the end of the string wto (that is, the address of the terminating
null character wto + strlen (wfrom) ) rather than the beginning.
This function is not part of ISO or POSIX but was found useful while developing the GNU C Library itself. The behavior of
|
char * stpncpy (char *restrict to, const char *restrict from, size_t size) | Function |
This function is similar to stpcpy but copies always exactly
size characters into to.
If the length of from is more then size, then If the length of from is less than size, then This function is not part of ISO or POSIX but was found useful while developing the GNU C Library itself. Its behavior is undefined if the strings overlap. The function is
declared in |
wchar_t * wcpncpy (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size) | Function |
This function is similar to wcpcpy but copies always exactly
wsize characters into wto.
If the length of wfrom is more then size, then
If the length of wfrom is less than size, then This function is not part of ISO or POSIX but was found useful while developing the GNU C Library itself. Its behavior is undefined if the strings overlap.
|
char * strdupa (const char *s) | Macro |
This macro is similar to strdup but allocates the new string
using alloca instead of malloc (see Variable Size Automatic). This means of course the returned string has the same
limitations as any block of memory allocated using alloca .
For obvious reasons #include <paths.h> #include <string.h> #include <stdio.h> const char path[] = _PATH_STDPATH; int main (void) { char *wr_path = strdupa (path); char *cp = strtok (wr_path, ":"); while (cp != NULL) { puts (cp); cp = strtok (NULL, ":"); } return 0; } Please note that calling This function is only available if GNU CC is used. |
char * strndupa (const char *s, size_t size) | Macro |
This function is similar to strndup but like strdupa it
allocates the new string using alloca
see Variable Size Automatic. The same advantages and limitations
of strdupa are valid for strndupa , too.
This function is implemented only as a macro, just like
|
char * strcat (char *restrict to, const char *restrict from) | Function |
The strcat function is similar to strcpy , except that the
characters from from are concatenated or appended to the end of
to, instead of overwriting it. That is, the first character from
from overwrites the null character marking the end of to.
An equivalent definition for char * strcat (char *restrict to, const char *restrict from) { strcpy (to + strlen (to), from); return to; } This function has undefined results if the strings overlap. |
wchar_t * wcscat (wchar_t *restrict wto, const wchar_t *restrict wfrom) | Function |
The wcscat function is similar to wcscpy , except that the
characters from wfrom are concatenated or appended to the end of
wto, instead of overwriting it. That is, the first character from
wfrom overwrites the null character marking the end of wto.
An equivalent definition for wchar_t * wcscat (wchar_t *wto, const wchar_t *wfrom) { wcscpy (wto + wcslen (wto), wfrom); return wto; } This function has undefined results if the strings overlap. |
Programmers using the strcat
or wcscat
function (or the
following strncat
or wcsncar
functions for that matter)
can easily be recognized as lazy and reckless. In almost all situations
the lengths of the participating strings are known (it better should be
since how can one otherwise ensure the allocated size of the buffer is
sufficient?) Or at least, one could know them if one keeps track of the
results of the various function calls. But then it is very inefficient
to use strcat
/wcscat
. A lot of time is wasted finding the
end of the destination string so that the actual copying can start.
This is a common example:
/* This function concatenates arbitrarily many strings. The last parameter must beNULL
. */ char * concat (const char *str, ...) { va_list ap, ap2; size_t total = 1; const char *s; char *result; va_start (ap, str); /* Actuallyva_copy
, but this is the name more gcc versions understand. */ __va_copy (ap2, ap); /* Determine how much space we need. */ for (s = str; s != NULL; s = va_arg (ap, const char *)) total += strlen (s); va_end (ap); result = (char *) malloc (total); if (result != NULL) { result[0] = '\0'; /* Copy the strings. */ for (s = str; s != NULL; s = va_arg (ap2, const char *)) strcat (result, s); } va_end (ap2); return result; }
This looks quite simple, especially the second loop where the strings
are actually copied. But these innocent lines hide a major performance
penalty. Just imagine that ten strings of 100 bytes each have to be
concatenated. For the second string we search the already stored 100
bytes for the end of the string so that we can append the next string.
For all strings in total the comparisons necessary to find the end of
the intermediate results sums up to 5500! If we combine the copying
with the search for the allocation we can write this function more
efficient:
char * concat (const char *str, ...) { va_list ap; size_t allocated = 100; char *result = (char *) malloc (allocated); char *wp; if (allocated != NULL) { char *newp; va_start (ap, atr); wp = result; for (s = str; s != NULL; s = va_arg (ap, const char *)) { size_t len = strlen (s); /* Resize the allocated memory if necessary. */ if (wp + len + 1 > result + allocated) { allocated = (allocated + len) * 2; newp = (char *) realloc (result, allocated); if (newp == NULL) { free (result); return NULL; } wp = newp + (wp - result); result = newp; } wp = mempcpy (wp, s, len); } /* Terminate the result string. */ *wp++ = '\0'; /* Resize memory to the optimal size. */ newp = realloc (result, wp - result); if (newp != NULL) result = newp; va_end (ap); } return result; }
With a bit more knowledge about the input strings one could fine-tune
the memory allocation. The difference we are pointing to here is that
we don't use strcat
anymore. We always keep track of the length
of the current intermediate result so we can safe us the search for the
end of the string and use mempcpy
. Please note that we also
don't use stpcpy
which might seem more natural since we handle
with strings. But this is not necessary since we already know the
length of the string and therefore can use the faster memory copying
function. The example would work for wide characters the same way.
Whenever a programmer feels the need to use strcat
she or he
should think twice and look through the program whether the code cannot
be rewritten to take advantage of already calculated results. Again: it
is almost always unnecessary to use strcat
.
char * strncat (char *restrict to, const char *restrict from, size_t size) | Function |
This function is like strcat except that not more than size
characters from from are appended to the end of to. A
single null character is also always appended to to, so the total
allocated size of to must be at least size + 1 bytes
longer than its initial length.
The char * strncat (char *to, const char *from, size_t size) { to[strlen (to) + size] = '\0'; strncpy (to + strlen (to), from, size); return to; } The behavior of |
wchar_t * wcsncat (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size) | Function |
This function is like wcscat except that not more than size
characters from from are appended to the end of to. A
single null character is also always appended to to, so the total
allocated size of to must be at least size + 1 bytes
longer than its initial length.
The wchar_t * wcsncat (wchar_t *restrict wto, const wchar_t *restrict wfrom, size_t size) { wto[wcslen (to) + size] = L'\0'; wcsncpy (wto + wcslen (wto), wfrom, size); return wto; } The behavior of |
Here is an example showing the use of strncpy
and strncat
(the wide character version is equivalent). Notice how, in the call to
strncat
, the size parameter is computed to avoid
overflowing the character array buffer
.
#include <string.h> #include <stdio.h> #define SIZE 10 static char buffer[SIZE]; main () { strncpy (buffer, "hello", SIZE); puts (buffer); strncat (buffer, ", world", SIZE - strlen (buffer) - 1); puts (buffer); }
The output produced by this program looks like:
hello hello, wo
void bcopy (const void *from, void *to, size_t size) | Function |
This is a partially obsolete alternative for memmove , derived from
BSD. Note that it is not quite equivalent to memmove , because the
arguments are not in the same order and there is no return value.
|
void bzero (void *block, size_t size) | Function |
This is a partially obsolete alternative for memset , derived from
BSD. Note that it is not as general as memset , because the only
value it can store is zero.
|
You can use the functions in this section to perform comparisons on the contents of strings and arrays. As well as checking for equality, these functions can also be used as the ordering functions for sorting operations. See Searching and Sorting, for an example of this.
Unlike most comparison operations in C, the string comparison functions return a nonzero value if the strings are not equivalent rather than if they are. The sign of the value indicates the relative ordering of the first characters in the strings that are not equivalent: a negative value indicates that the first string is "less" than the second, while a positive value indicates that the first string is "greater".
The most common use of these functions is to check only for equality.
This is canonically done with an expression like ! strcmp (s1, s2)
.
All of these functions are declared in the header file string.h
.
int memcmp (const void *a1, const void *a2, size_t size) | Function |
The function memcmp compares the size bytes of memory
beginning at a1 against the size bytes of memory beginning
at a2. The value returned has the same sign as the difference
between the first differing pair of bytes (interpreted as unsigned
char objects, then promoted to int ).
If the contents of the two blocks are equal, |
int wmemcmp (const wchar_t *a1, const wchar_t *a2, size_t size) | Function |
The function wmemcmp compares the size wide characters
beginning at a1 against the size wide characters beginning
at a2. The value returned is smaller than or larger than zero
depending on whether the first differing wide character is a1 is
smaller or larger than the corresponding character in a2.
If the contents of the two blocks are equal, |
On arbitrary arrays, the memcmp
function is mostly useful for
testing equality. It usually isn't meaningful to do byte-wise ordering
comparisons on arrays of things other than bytes. For example, a
byte-wise comparison on the bytes that make up floating-point numbers
isn't likely to tell you anything about the relationship between the
values of the floating-point numbers.
wmemcmp
is really only useful to compare arrays of type
wchar_t
since the function looks at sizeof (wchar_t)
bytes
at a time and this number of bytes is system dependent.
You should also be careful about using memcmp
to compare objects
that can contain "holes", such as the padding inserted into structure
objects to enforce alignment requirements, extra space at the end of
unions, and extra characters at the ends of strings whose length is less
than their allocated size. The contents of these "holes" are
indeterminate and may cause strange behavior when performing byte-wise
comparisons. For more predictable results, perform an explicit
component-wise comparison.
For example, given a structure type definition like:
struct foo { unsigned char tag; union { double f; long i; char *p; } value; };
you are better off writing a specialized comparison function to compare
struct foo
objects instead of comparing them with memcmp
.
int strcmp (const char *s1, const char *s2) | Function |
The strcmp function compares the string s1 against
s2, returning a value that has the same sign as the difference
between the first differing pair of characters (interpreted as
unsigned char objects, then promoted to int ).
If the two strings are equal, A consequence of the ordering used by
|
int wcscmp (const wchar_t *ws1, const wchar_t *ws2) | Function |
The If the two strings are equal, A consequence of the ordering used by
|
int strcasecmp (const char *s1, const char *s2) | Function |
This function is like strcmp , except that differences in case are
ignored. How uppercase and lowercase characters are related is
determined by the currently selected locale. In the standard "C"
locale the characters Ä and ä do not match but in a locale which
regards these characters as parts of the alphabet they do match.
|
int wcscasecmp (const wchar_t *ws1, const wchar_T *ws2) | Function |
This function is like wcscmp , except that differences in case are
ignored. How uppercase and lowercase characters are related is
determined by the currently selected locale. In the standard "C"
locale the characters Ä and ä do not match but in a locale which
regards these characters as parts of the alphabet they do match.
|
int strncmp (const char *s1, const char *s2, size_t size) | Function |
This function is the similar to strcmp , except that no more than
size wide characters are compared. In other words, if the two
strings are the same in their first size wide characters, the
return value is zero.
|
int wcsncmp (const wchar_t *ws1, const wchar_t *ws2, size_t size) | Function |
This function is the similar to wcscmp , except that no more than
size wide characters are compared. In other words, if the two
strings are the same in their first size wide characters, the
return value is zero.
|
int strncasecmp (const char *s1, const char *s2, size_t n) | Function |
This function is like strncmp , except that differences in case
are ignored. Like strcasecmp , it is locale dependent how
uppercase and lowercase characters are related.
|
int wcsncasecmp (const wchar_t *ws1, const wchar_t *s2, size_t n) | Function |
This function is like wcsncmp , except that differences in case
are ignored. Like wcscasecmp , it is locale dependent how
uppercase and lowercase characters are related.
|
Here are some examples showing the use of strcmp
and
strncmp
(equivalent examples can be constructed for the wide
character functions). These examples assume the use of the ASCII
character set. (If some other character set--say, EBCDIC--is used
instead, then the glyphs are associated with different numeric codes,
and the return values and ordering may differ.)
strcmp ("hello", "hello") => 0 /* These two strings are the same. */ strcmp ("hello", "Hello") => 32 /* Comparisons are case-sensitive. */ strcmp ("hello", "world") => -15 /* The character'h'
comes before'w'
. */ strcmp ("hello", "hello, world") => -44 /* Comparing a null character against a comma. */ strncmp ("hello", "hello, world", 5) => 0 /* The initial 5 characters are the same. */ strncmp ("hello, world", "hello, stupid world!!!", 5) => 0 /* The initial 5 characters are the same. */
int strverscmp (const char *s1, const char *s2) | Function |
The strverscmp function compares the string s1 against
s2, considering them as holding indices/version numbers. Return
value follows the same conventions as found in the strverscmp
function. In fact, if s1 and s2 contain no digits,
strverscmp behaves like strcmp .
Basically, we compare strings normally (character by character), until we find a digit in each string - then we enter a special comparison mode, where each sequence of digits is taken as a whole. If we reach the end of these two parts without noticing a difference, we return to the standard comparison mode. There are two types of numeric parts: "integral" and "fractional" (those begin with a '0'). The types of the numeric parts affect the way we sort them:
strverscmp ("no digit", "no digit") => 0 /* same behavior as strcmp. */ strverscmp ("item#99", "item#100") => <0 /* same prefix, but 99 < 100. */ strverscmp ("alpha1", "alpha001") => >0 /* fractional part inferior to integral one. */ strverscmp ("part1_f012", "part1_f01") => >0 /* two fractional parts. */ strverscmp ("foo.009", "foo.0") => <0 /* idem, but with leading zeroes only. */ This function is especially useful when dealing with filename sorting, because filenames frequently hold indices/version numbers.
|
int bcmp (const void *a1, const void *a2, size_t size) | Function |
This is an obsolete alias for memcmp , derived from BSD.
|
In some locales, the conventions for lexicographic ordering differ from
the strict numeric ordering of character codes. For example, in Spanish
most glyphs with diacritical marks such as accents are not considered
distinct letters for the purposes of collation. On the other hand, the
two-character sequence ll
is treated as a single letter that is
collated immediately after l
.
You can use the functions strcoll
and strxfrm
(declared in
the headers file string.h
) and wcscoll
and wcsxfrm
(declared in the headers file wchar
) to compare strings using a
collation ordering appropriate for the current locale. The locale used
by these functions in particular can be specified by setting the locale
for the LC_COLLATE
category; see Locales.
In the standard C locale, the collation sequence for strcoll
is
the same as that for strcmp
. Similarly, wcscoll
and
wcscmp
are the same in this situation.
Effectively, the way these functions work is by applying a mapping to transform the characters in a string to a byte sequence that represents the string's position in the collating sequence of the current locale. Comparing two such byte sequences in a simple fashion is equivalent to comparing the strings with the locale's collating sequence.
The functions strcoll
and wcscoll
perform this translation
implicitly, in order to do one comparison. By contrast, strxfrm
and wcsxfrm
perform the mapping explicitly. If you are making
multiple comparisons using the same string or set of strings, it is
likely to be more efficient to use strxfrm
or wcsxfrm
to
transform all the strings just once, and subsequently compare the
transformed strings with strcmp
or wcscmp
.
int strcoll (const char *s1, const char *s2) | Function |
The strcoll function is similar to strcmp but uses the
collating sequence of the current locale for collation (the
LC_COLLATE locale).
|
int wcscoll (const wchar_t *ws1, const wchar_t *ws2) | Function |
The wcscoll function is similar to wcscmp but uses the
collating sequence of the current locale for collation (the
LC_COLLATE locale).
|
Here is an example of sorting an array of strings, using strcoll
to compare them. The actual sort algorithm is not written here; it
comes from qsort
(see Array Sort Function). The job of the
code shown here is to say how to compare the strings while sorting them.
(Later on in this section, we will show a way to do this more
efficiently using strxfrm
.)
/* This is the comparison function used withqsort
. */ int compare_elements (char **p1, char **p2) { return strcoll (*p1, *p2); } /* This is the entry point--the function to sort strings using the locale's collating sequence. */ void sort_strings (char **array, int nstrings) { /* Sorttemp_array
by comparing the strings. */ qsort (array, nstrings, sizeof (char *), compare_elements); }
size_t strxfrm (char *restrict to, const char *restrict from, size_t size) | Function |
The function strxfrm transforms the string from using the
collation transformation determined by the locale currently selected for
collation, and stores the transformed string in the array to. Up
to size characters (including a terminating null character) are
stored.
The behavior is undefined if the strings to and from overlap; see Copying and Concatenation. The return value is the length of the entire transformed string. This
value is not affected by the value of size, but if it is greater
or equal than size, it means that the transformed string did not
entirely fit in the array to. In this case, only as much of the
string as actually fits was stored. To get the whole transformed
string, call The transformed string may be longer than the original string, and it may also be shorter. If size is zero, no characters are stored in to. In this
case, |
size_t wcsxfrm (wchar_t *restrict wto, const wchar_t *wfrom, size_t size) | Function |
The function wcsxfrm transforms wide character string wfrom
using the collation transformation determined by the locale currently
selected for collation, and stores the transformed string in the array
wto. Up to size wide characters (including a terminating null
character) are stored.
The behavior is undefined if the strings wto and wfrom overlap; see Copying and Concatenation. The return value is the length of the entire transformed wide character
string. This value is not affected by the value of size, but if
it is greater or equal than size, it means that the transformed
wide character string did not entirely fit in the array wto. In
this case, only as much of the wide character string as actually fits
was stored. To get the whole transformed wide character string, call
The transformed wide character string may be longer than the original wide character string, and it may also be shorter. If size is zero, no characters are stored in to. In this
case, |
Here is an example of how you can use strxfrm
when
you plan to do many comparisons. It does the same thing as the previous
example, but much faster, because it has to transform each string only
once, no matter how many times it is compared with other strings. Even
the time needed to allocate and free storage is much less than the time
we save, when there are many strings.
struct sorter { char *input; char *transformed; }; /* This is the comparison function used withqsort
to sort an array ofstruct sorter
. */ int compare_elements (struct sorter *p1, struct sorter *p2) { return strcmp (p1->transformed, p2->transformed); } /* This is the entry point--the function to sort strings using the locale's collating sequence. */ void sort_strings_fast (char **array, int nstrings) { struct sorter temp_array[nstrings]; int i; /* Set uptemp_array
. Each element contains one input string and its transformed string. */ for (i = 0; i < nstrings; i++) { size_t length = strlen (array[i]) * 2; char *transformed; size_t transformed_length; temp_array[i].input = array[i]; /* First try a buffer perhaps big enough. */ transformed = (char *) xmalloc (length); /* Transformarray[i]
. */ transformed_length = strxfrm (transformed, array[i], length); /* If the buffer was not large enough, resize it and try again. */ if (transformed_length >= length) { /* Allocate the needed space. +1 for terminatingNUL
character. */ transformed = (char *) xrealloc (transformed, transformed_length + 1); /* The return value is not interesting because we know how long the transformed string is. */ (void) strxfrm (transformed, array[i], transformed_length + 1); } temp_array[i].transformed = transformed; } /* Sorttemp_array
by comparing transformed strings. */ qsort (temp_array, sizeof (struct sorter), nstrings, compare_elements); /* Put the elements back in the permanent array in their sorted order. */ for (i = 0; i < nstrings; i++) array[i] = temp_array[i].input; /* Free the strings we allocated. */ for (i = 0; i < nstrings; i++) free (temp_array[i].transformed); }
The interesting part of this code for the wide character version would
look like this:
void sort_strings_fast (wchar_t **array, int nstrings) { ... /* Transformarray[i]
. */ transformed_length = wcsxfrm (transformed, array[i], length); /* If the buffer was not large enough, resize it and try again. */ if (transformed_length >= length) { /* Allocate the needed space. +1 for terminatingNUL
character. */ transformed = (wchar_t *) xrealloc (transformed, (transformed_length + 1) * sizeof (wchar_t)); /* The return value is not interesting because we know how long the transformed string is. */ (void) wcsxfrm (transformed, array[i], transformed_length + 1); } ...
Note the additional multiplication with sizeof (wchar_t)
in the
realloc
call.
Compatibility Note: The string collation functions are a new feature of ISO C90. Older C dialects have no equivalent feature. The wide character versions were introduced in Amendment 1 to ISO C90.
This section describes library functions which perform various kinds
of searching operations on strings and arrays. These functions are
declared in the header file string.h
.
void * memchr (const void *block, int c, size_t size) | Function |
This function finds the first occurrence of the byte c (converted
to an unsigned char ) in the initial size bytes of the
object beginning at block. The return value is a pointer to the
located byte, or a null pointer if no match was found.
|
wchar_t * wmemchr (const wchar_t *block, wchar_t wc, size_t size) | Function |
This function finds the first occurrence of the wide character wc in the initial size wide characters of the object beginning at block. The return value is a pointer to the located wide character, or a null pointer if no match was found. |
void * rawmemchr (const void *block, int c) | Function |
Often the memchr function is used with the knowledge that the
byte c is available in the memory block specified by the
parameters. But this means that the size parameter is not really
needed and that the tests performed with it at runtime (to check whether
the end of the block is reached) are not needed.
The This function is of special interest when looking for the end of a
string. Since all strings are terminated by a null byte a call like
rawmemchr (str, '\0') will never go beyond the end of the string. This function is a GNU extension. |
void * memrchr (const void *block, int c, size_t size) | Function |
The function memrchr is like memchr , except that it searches
backwards from the end of the block defined by block and size
(instead of forwards from the front).
|
char * strchr (const char *string, int c) | Function |
The strchr function finds the first occurrence of the character
c (converted to a char ) in the null-terminated string
beginning at string. The return value is a pointer to the located
character, or a null pointer if no match was found.
For example,
strchr ("hello, world", 'l') => "llo, world" strchr ("hello, world", '?') => NULL The terminating null character is considered to be part of the string,
so you can use this function get a pointer to the end of a string by
specifying a null character as the value of the c argument. It
would be better (but less portable) to use |
wchar_t * wcschr (const wchar_t *wstring, int wc) | Function |
The wcschr function finds the first occurrence of the wide
character wc in the null-terminated wide character string
beginning at wstring. The return value is a pointer to the
located wide character, or a null pointer if no match was found.
The terminating null character is considered to be part of the wide
character string, so you can use this function get a pointer to the end
of a wide character string by specifying a null wude character as the
value of the wc argument. It would be better (but less portable)
to use |
char * strchrnul (const char *string, int c) | Function |
strchrnul is the same as strchr except that if it does
not find the character, it returns a pointer to string's terminating
null character rather than a null pointer.
This function is a GNU extension. |
wchar_t * wcschrnul (const wchar_t *wstring, wchar_t wc) | Function |
wcschrnul is the same as wcschr except that if it does not
find the wide character, it returns a pointer to wide character string's
terminating null wide character rather than a null pointer.
This function is a GNU extension. |
One useful, but unusual, use of the strchr
function is when one wants to have a pointer pointing to the NUL byte
terminating a string. This is often written in this way:
s += strlen (s);
This is almost optimal but the addition operation duplicated a bit of
the work already done in the strlen
function. A better solution
is this:
s = strchr (s, '\0');
There is no restriction on the second parameter of strchr
so it
could very well also be the NUL character. Those readers thinking very
hard about this might now point out that the strchr
function is
more expensive than the strlen
function since we have two abort
criteria. This is right. But in the GNU C library the implementation of
strchr
is optimized in a special way so that strchr
actually is faster.
char * strrchr (const char *string, int c) | Function |
The function strrchr is like strchr , except that it searches
backwards from the end of the string string (instead of forwards
from the front).
For example,
strrchr ("hello, world", 'l') => "ld" |
wchar_t * wcsrchr (const wchar_t *wstring, wchar_t c) | Function |
The function wcsrchr is like wcschr , except that it searches
backwards from the end of the string wstring (instead of forwards
from the front).
|
char * strstr (const char *haystack, const char *needle) | Function |
This is like strchr , except that it searches haystack for a
substring needle rather than just a single character. It
returns a pointer into the string haystack that is the first
character of the substring, or a null pointer if no match was found. If
needle is an empty string, the function returns haystack.
For example,
strstr ("hello, world", "l") => "llo, world" strstr ("hello, world", "wo") => "world" |
wchar_t * wcsstr (const wchar_t *haystack, const wchar_t *needle) | Function |
This is like wcschr , except that it searches haystack for a
substring needle rather than just a single wide character. It
returns a pointer into the string haystack that is the first wide
character of the substring, or a null pointer if no match was found. If
needle is an empty string, the function returns haystack.
|
wchar_t * wcswcs (const wchar_t *haystack, const wchar_t *needle) | Function |
wcsstr is an depricated alias for wcsstr . This is the
name originally used in the X/Open Portability Guide before the
Amendment 1 to ISO C90 was published.
|
char * strcasestr (const char *haystack, const char *needle) | Function |
This is like strstr , except that it ignores case in searching for
the substring. Like strcasecmp , it is locale dependent how
uppercase and lowercase characters are related.
For example,
strstr ("hello, world", "L") => "llo, world" strstr ("hello, World", "wo") => "World" |
void * memmem (const void *haystack, size_t haystack-len, const void *needle, size_t needle-len) |
Function |
This is like strstr , but needle and haystack are byte
arrays rather than null-terminated strings. needle-len is the
length of needle and haystack-len is the length of
haystack.
This function is a GNU extension. |
size_t strspn (const char *string, const char *skipset) | Function |
The strspn ("string span") function returns the length of the
initial substring of string that consists entirely of characters that
are members of the set specified by the string skipset. The order
of the characters in skipset is not important.
For example,
strspn ("hello, world", "abcdefghijklmnopqrstuvwxyz") => 5 Note that "character" is here used in the sense of byte. In a string using a multibyte character encoding (abstract) character consisting of more than one byte are not treated as an entity. Each byte is treated separately. The function is not locale-dependent. |
size_t wcsspn (const wchar_t *wstring, const wchar_t *skipset) | Function |
The wcsspn ("wide character string span") function returns the
length of the initial substring of wstring that consists entirely
of wide characters that are members of the set specified by the string
skipset. The order of the wide characters in skipset is not
important.
|
size_t strcspn (const char *string, const char *stopset) | Function |
The strcspn ("string complement span") function returns the length
of the initial substring of string that consists entirely of characters
that are not members of the set specified by the string stopset.
(In other words, it returns the offset of the first character in string
that is a member of the set stopset.)
For example,
strcspn ("hello, world", " \t\n,.;!?") => 5 Note that "character" is here used in the sense of byte. In a string using a multibyte character encoding (abstract) character consisting of more than one byte are not treated as an entity. Each byte is treated separately. The function is not locale-dependent. |
size_t wcscspn (const wchar_t *wstring, const wchar_t *stopset) | Function |
The wcscspn ("wide character string complement span") function
returns the length of the initial substring of wstring that
consists entirely of wide characters that are not members of the
set specified by the string stopset. (In other words, it returns
the offset of the first character in string that is a member of
the set stopset.)
|
char * strpbrk (const char *string, const char *stopset) | Function |
The strpbrk ("string pointer break") function is related to
strcspn , except that it returns a pointer to the first character
in string that is a member of the set stopset instead of the
length of the initial substring. It returns a null pointer if no such
character from stopset is found.
For example,
strpbrk ("hello, world", " \t\n,.;!?") => ", world" Note that "character" is here used in the sense of byte. In a string using a multibyte character encoding (abstract) character consisting of more than one byte are not treated as an entity. Each byte is treated separately. The function is not locale-dependent. |
wchar_t * wcspbrk (const wchar_t *wstring, const wchar_t *stopset) | Function |
The wcspbrk ("wide character string pointer break") function is
related to wcscspn , except that it returns a pointer to the first
wide character in wstring that is a member of the set
stopset instead of the length of the initial substring. It
returns a null pointer if no such character from stopset is found.
|
char * index (const char *string, int c) | Function |
index is another name for strchr ; they are exactly the same.
New code should always use strchr since this name is defined in
ISO C while index is a BSD invention which never was available
on System V derived systems.
|
char * rindex (const char *string, int c) | Function |
rindex is another name for strrchr ; they are exactly the same.
New code should always use strrchr since this name is defined in
ISO C while rindex is a BSD invention which never was available
on System V derived systems.
|
It's fairly common for programs to have a need to do some simple kinds
of lexical analysis and parsing, such as splitting a command string up
into tokens. You can do this with the strtok
function, declared
in the header file string.h
.
char * strtok (char *restrict newstring, const char *restrict delimiters) | Function |
A string can be split into tokens by making a series of calls to the
function strtok .
The string to be split up is passed as the newstring argument on
the first call only. The The delimiters argument is a string that specifies a set of delimiters that may surround the token being extracted. All the initial characters that are members of this set are discarded. The first character that is not a member of this set of delimiters marks the beginning of the next token. The end of the token is found by looking for the next character that is a member of the delimiter set. This character in the original string newstring is overwritten by a null character, and the pointer to the beginning of the token in newstring is returned. On the next call to If the end of the string newstring is reached, or if the remainder of
string consists only of delimiter characters, Note that "character" is here used in the sense of byte. In a string using a multibyte character encoding (abstract) character consisting of more than one byte are not treated as an entity. Each byte is treated separately. The function is not locale-dependent. Note that "character" is here used in the sense of byte. In a string using a multibyte character encoding (abstract) character consisting of more than one byte are not treated as an entity. Each byte is treated separately. The function is not locale-dependent. |
wchar_t * wcstok (wchar_t *newstring, const char *delimiters) | Function |
A string can be split into tokens by making a series of calls to the
function wcstok .
The string to be split up is passed as the newstring argument on
the first call only. The The delimiters argument is a wide character string that specifies a set of delimiters that may surround the token being extracted. All the initial wide characters that are members of this set are discarded. The first wide character that is not a member of this set of delimiters marks the beginning of the next token. The end of the token is found by looking for the next wide character that is a member of the delimiter set. This wide character in the original wide character string newstring is overwritten by a null wide character, and the pointer to the beginning of the token in newstring is returned. On the next call to If the end of the wide character string newstring is reached, or
if the remainder of string consists only of delimiter wide characters,
Note that "character" is here used in the sense of byte. In a string using a multibyte character encoding (abstract) character consisting of more than one byte are not treated as an entity. Each byte is treated separately. The function is not locale-dependent. |
Warning: Since strtok
and wcstok
alter the string
they is parsing, you should always copy the string to a temporary buffer
before parsing it with strtok
/wcstok
(see Copying and Concatenation). If you allow strtok
or wcstok
to modify
a string that came from another part of your program, you are asking for
trouble; that string might be used for other purposes after
strtok
or wcstok
has modified it, and it would not have
the expected value.
The string that you are operating on might even be a constant. Then
when strtok
or wcstok
tries to modify it, your program
will get a fatal signal for writing in read-only memory. See Program Error Signals. Even if the operation of strtok
or wcstok
would not require a modification of the string (e.g., if there is
exactly one token) the string can (and in the GNU libc case will) be
modified.
This is a special case of a general principle: if a part of a program does not have as its purpose the modification of a certain data structure, then it is error-prone to modify the data structure temporarily.
The functions strtok
and wcstok
are not reentrant.
See Nonreentrancy, for a discussion of where and why reentrancy is
important.
Here is a simple example showing the use of strtok
.
#include <string.h> #include <stddef.h> ... const char string[] = "words separated by spaces -- and, punctuation!"; const char delimiters[] = " .,;:!-"; char *token, *cp; ... cp = strdupa (string); /* Make writable copy. */ token = strtok (cp, delimiters); /* token => "words" */ token = strtok (NULL, delimiters); /* token => "separated" */ token = strtok (NULL, delimiters); /* token => "by" */ token = strtok (NULL, delimiters); /* token => "spaces" */ token = strtok (NULL, delimiters); /* token => "and" */ token = strtok (NULL, delimiters); /* token => "punctuation" */ token = strtok (NULL, delimiters); /* token => NULL */
The GNU C library contains two more functions for tokenizing a string which overcome the limitation of non-reentrancy. They are only available for multibyte character strings.
char * strtok_r (char *newstring, const char *delimiters, char **save_ptr) | Function |
Just like strtok , this function splits the string into several
tokens which can be accessed by successive calls to strtok_r .
The difference is that the information about the next token is stored in
the space pointed to by the third argument, save_ptr, which is a
pointer to a string pointer. Calling strtok_r with a null
pointer for newstring and leaving save_ptr between the calls
unchanged does the job without hindering reentrancy.
This function is defined in POSIX.1 and can be found on many systems which support multi-threading. |
char * strsep (char **string_ptr, const char *delimiter) | Function |
This function has a similar functionality as strtok_r with the
newstring argument replaced by the save_ptr argument. The
initialization of the moving pointer has to be done by the user.
Successive calls to strsep move the pointer along the tokens
separated by delimiter, returning the address of the next token
and updating string_ptr to point to the beginning of the next
token.
One difference between This function was introduced in 4.3BSD and therefore is widely available. |
Here is how the above example looks like when strsep
is used.
#include <string.h> #include <stddef.h> ... const char string[] = "words separated by spaces -- and, punctuation!"; const char delimiters[] = " .,;:!-"; char *running; char *token; ... running = strdupa (string); token = strsep (&running, delimiters); /* token => "words" */ token = strsep (&running, delimiters); /* token => "separated" */ token = strsep (&running, delimiters); /* token => "by" */ token = strsep (&running, delimiters); /* token => "spaces" */ token = strsep (&running, delimiters); /* token => "" */ token = strsep (&running, delimiters); /* token => "" */ token = strsep (&running, delimiters); /* token => "" */ token = strsep (&running, delimiters); /* token => "and" */ token = strsep (&running, delimiters); /* token => "" */ token = strsep (&running, delimiters); /* token => "punctuation" */ token = strsep (&running, delimiters); /* token => "" */ token = strsep (&running, delimiters); /* token => NULL */
char * basename (const char *filename) | Function |
The GNU version of the basename function returns the last
component of the path in filename. This function is the preferred
usage, since it does not modify the argument, filename, and
respects trailing slashes. The prototype for basename can be
found in string.h . Note, this function is overriden by the XPG
version, if libgen.h is included.
Example of using GNU #include <string.h> int main (int argc, char *argv[]) { char *prog = basename (argv[0]); if (argc < 2) { fprintf (stderr, "Usage %s <arg>\n", prog); exit (1); } ... } Portability Note: This function may produce different results on different systems. |
char * basename (char *path) | Function |
This is the standard XPG defined basename . It is similar in
spirit to the GNU version, but may modify the path by removing
trailing '/' characters. If the path is made up entirely of '/'
characters, then "/" will be returned. Also, if path is
NULL or an empty string, then "." is returned. The prototype for
the XPG version can be found in libgen.h .
Example of using XPG #include <libgen.h> int main (int argc, char *argv[]) { char *prog; char *path = strdupa (argv[0]); prog = basename (path); if (argc < 2) { fprintf (stderr, "Usage %s <arg>\n", prog); exit (1); } ... } |
char * dirname (char *path) | Function |
The dirname function is the compliment to the XPG version of
basename . It returns the parent directory of the file specified
by path. If path is NULL , an empty string, or
contains no '/' characters, then "." is returned. The prototype for this
function can be found in libgen.h .
|
The function below addresses the perennial programming quandary: "How do
I take good data in string form and painlessly turn it into garbage?"
This is actually a fairly simple task for C programmers who do not use
the GNU C library string functions, but for programs based on the GNU C
library, the strfry
function is the preferred method for
destroying string data.
The prototype for this function is in string.h
.
char * strfry (char *string) | Function |
The return value of Portability Note: This function is unique to the GNU C library. |
The memfrob
function converts an array of data to something
unrecognizable and back again. It is not encryption in its usual sense
since it is easy for someone to convert the encrypted data back to clear
text. The transformation is analogous to Usenet's "Rot13" encryption
method for obscuring offensive jokes from sensitive eyes and such.
Unlike Rot13, memfrob
works on arbitrary binary data, not just
text.
For true encryption, See Cryptographic Functions.
This function is declared in string.h
.
void * memfrob (void *mem, size_t length) | Function |
Note that This is a good function for hiding information from someone who doesn't want to see it or doesn't want to see it very much. To really prevent people from retrieving the information, use stronger encryption such as that described in See Cryptographic Functions. Portability Note: This function is unique to the GNU C library. |
To store or transfer binary data in environments which only support text one has to encode the binary data by mapping the input bytes to characters in the range allowed for storing or transfering. SVID systems (and nowadays XPG compliant systems) provide minimal support for this task.
char * l64a (long int n) | Function |
This function encodes a 32-bit input value using characters from the
basic character set. It returns a pointer to a 6 character buffer which
contains an encoded version of n. To encode a series of bytes the
user must copy the returned string to a destination buffer. It returns
the empty string if n is zero, which is somewhat bizarre but
mandated by the standard. Warning: Since a static buffer is used this function should not be used in multi-threaded programs. There is no thread-safe alternative to this function in the C library. Compatibility Note: The XPG standard states that the return value of l64a is undefined if n is negative. In the GNU
implementation, l64a treats its argument as unsigned, so it will
return a sensible encoding for any nonzero n; however, portable
programs should not rely on this.
To encode a large buffer char * encode (const void *buf, size_t len) { /* We know in advance how long the buffer has to be. */ unsigned char *in = (unsigned char *) buf; char *out = malloc (6 + ((len + 3) / 4) * 6 + 1); char *cp = out; /* Encode the length. */ /* Using `htonl' is necessary so that the data can be decoded even on machines with different byte order. */ cp = mempcpy (cp, l64a (htonl (len)), 6); while (len > 3) { unsigned long int n = *in++; n = (n << 8) | *in++; n = (n << 8) | *in++; n = (n << 8) | *in++; len -= 4; if (n) cp = mempcpy (cp, l64a (htonl (n)), 6); else /* `l64a' returns the empty string for n==0, so we must generate its encoding ("......") by hand. */ cp = stpcpy (cp, "......"); } if (len > 0) { unsigned long int n = *in++; if (--len > 0) { n = (n << 8) | *in++; if (--len > 0) n = (n << 8) | *in; } memcpy (cp, l64a (htonl (n)), 6); cp += 6; } *cp = '\0'; return out; } It is strange that the library does not provide the complete functionality needed but so be it. |
To decode data produced with l64a
the following function should be
used.
long int a64l (const char *string) | Function |
The parameter string should contain a string which was produced by
a call to l64a . The function processes at least 6 characters of
this string, and decodes the characters it finds according to the table
below. It stops decoding when it finds a character not in the table,
rather like atoi ; if you have a buffer which has been broken into
lines, you must be careful to skip over the end-of-line characters.
The decoded number is returned as a |
The l64a
and a64l
functions use a base 64 encoding, in
which each character of an encoded string represents six bits of an
input word. These symbols are used for the base 64 digits:
0 | 1 | 2 | 3 | 4 | 5 | 6 | 7
| |
0 | . | / | 0 | 1
| 2 | 3 | 4 | 5
|
8 | 6 | 7 | 8 | 9
| A | B | C | D
|
16 | E | F | G | H
| I | J | K | L
|
24 | M | N | O | P
| Q | R | S | T
|
32 | U | V | W | X
| Y | Z | a | b
|
40 | c | d | e | f
| g | h | i | j
|
48 | k | l | m | n
| o | p | q | r
|
56 | s | t | u | v
| w | x | y | z
|
This encoding scheme is not standard. There are some other encoding methods which are much more widely used (UU encoding, MIME encoding). Generally, it is better to use one of these encodings.
argz vectors are vectors of strings in a contiguous block of
memory, each element separated from its neighbors by null-characters
('\0'
).
Envz vectors are an extension of argz vectors where each element is a
name-value pair, separated by a '='
character (as in a Unix
environment).
Each argz vector is represented by a pointer to the first element, of
type char *
, and a size, of type size_t
, both of which can
be initialized to 0
to represent an empty argz vector. All argz
functions accept either a pointer and a size argument, or pointers to
them, if they will be modified.
The argz functions use malloc
/realloc
to allocate/grow
argz vectors, and so any argz vector creating using these functions may
be freed by using free
; conversely, any argz function that may
grow a string expects that string to have been allocated using
malloc
(those argz functions that only examine their arguments or
modify them in place will work on any sort of memory).
See Unconstrained Allocation.
All argz functions that do memory allocation have a return type of
error_t
, and return 0
for success, and ENOMEM
if an
allocation error occurs.
These functions are declared in the standard include file argz.h
.
error_t argz_create (char *const argv[], char **argz, size_t *argz_len) | Function |
The argz_create function converts the Unix-style argument vector
argv (a vector of pointers to normal C strings, terminated by
(char *)0 ; see Program Arguments) into an argz vector with
the same elements, which is returned in argz and argz_len.
|
error_t argz_create_sep (const char *string, int sep, char **argz, size_t *argz_len) | Function |
The argz_create_sep function converts the null-terminated string
string into an argz vector (returned in argz and
argz_len) by splitting it into elements at every occurrence of the
character sep.
|
size_t argz_count (const char *argz, size_t arg_len) | Function |
Returns the number of elements in the argz vector argz and argz_len. |
void argz_extract (char *argz, size_t argz_len, char **argv) | Function |
The argz_extract function converts the argz vector argz and
argz_len into a Unix-style argument vector stored in argv,
by putting pointers to every element in argz into successive
positions in argv, followed by a terminator of 0 .
Argv must be pre-allocated with enough space to hold all the
elements in argz plus the terminating (char *)0
((argz_count (argz, argz_len) + 1) * sizeof (char *)
bytes should be enough). Note that the string pointers stored into
argv point into argz--they are not copies--and so
argz must be copied if it will be changed while argv is
still active. This function is useful for passing the elements in
argz to an exec function (see Executing a File).
|
void argz_stringify (char *argz, size_t len, int sep) | Function |
The argz_stringify converts argz into a normal string with
the elements separated by the character sep, by replacing each
'\0' inside argz (except the last one, which terminates the
string) with sep. This is handy for printing argz in a
readable manner.
|
error_t argz_add (char **argz, size_t *argz_len, const char *str) | Function |
The argz_add function adds the string str to the end of the
argz vector *argz , and updates *argz and
*argz_len accordingly.
|
error_t argz_add_sep (char **argz, size_t *argz_len, const char *str, int delim) | Function |
The argz_add_sep function is similar to argz_add , but
str is split into separate elements in the result at occurrences of
the character delim. This is useful, for instance, for
adding the components of a Unix search path to an argz vector, by using
a value of ':' for delim.
|
error_t argz_append (char **argz, size_t *argz_len, const char *buf, size_t buf_len) | Function |
The argz_append function appends buf_len bytes starting at
buf to the argz vector *argz , reallocating
*argz to accommodate it, and adding buf_len to
*argz_len .
|
error_t argz_delete (char **argz, size_t *argz_len, char *entry) | Function |
If entry points to the beginning of one of the elements in the
argz vector *argz , the argz_delete function will
remove this entry and reallocate *argz , modifying
*argz and *argz_len accordingly. Note that as
destructive argz functions usually reallocate their argz argument,
pointers into argz vectors such as entry will then become invalid.
|
error_t argz_insert (char **argz, size_t *argz_len, char *before, const char *entry) | Function |
The argz_insert function inserts the string entry into the
argz vector *argz at a point just before the existing
element pointed to by before, reallocating *argz and
updating *argz and *argz_len . If before
is 0 , entry is added to the end instead (as if by
argz_add ). Since the first element is in fact the same as
*argz , passing in *argz as the value of
before will result in entry being inserted at the beginning.
|
char * argz_next (char *argz, size_t argz_len, const char *entry) | Function |
The argz_next function provides a convenient way of iterating
over the elements in the argz vector argz. It returns a pointer
to the next element in argz after the element entry, or
0 if there are no elements following entry. If entry
is 0 , the first element of argz is returned.
This behavior suggests two styles of iteration:
char *entry = 0; while ((entry = argz_next (argz, argz_len, entry))) action; (the double parentheses are necessary to make some C compilers shut up
about what they consider a questionable char *entry; for (entry = argz; entry; entry = argz_next (argz, argz_len, entry)) action; Note that the latter depends on argz having a value of |
error_t argz_replace (char **argz, size_t *argz_len, const char *str, const char *with, unsigned *replace_count) | Function |
Replace any occurrences of the string str in argz with
with, reallocating argz as necessary. If
replace_count is non-zero, *replace_count will be
incremented by number of replacements performed.
|
Envz vectors are just argz vectors with additional constraints on the form of each element; as such, argz functions can also be used on them, where it makes sense.
Each element in an envz vector is a name-value pair, separated by a '='
character; if multiple '='
characters are present in an element, those
after the first are considered part of the value, and treated like all other
non-'\0'
characters.
If no '='
characters are present in an element, that element is
considered the name of a "null" entry, as distinct from an entry with an
empty value: envz_get
will return 0
if given the name of null
entry, whereas an entry with an empty value would result in a value of
""
; envz_entry
will still find such entries, however. Null
entries can be removed with envz_strip
function.
As with argz functions, envz functions that may allocate memory (and thus
fail) have a return type of error_t
, and return either 0
or
ENOMEM
.
These functions are declared in the standard include file envz.h
.
char * envz_entry (const char *envz, size_t envz_len, const char *name) | Function |
The envz_entry function finds the entry in envz with the name
name, and returns a pointer to the whole entry--that is, the argz
element which begins with name followed by a '=' character. If
there is no entry with that name, 0 is returned.
|
char * envz_get (const char *envz, size_t envz_len, const char *name) | Function |
The envz_get function finds the entry in envz with the name
name (like envz_entry ), and returns a pointer to the value
portion of that entry (following the '=' ). If there is no entry with
that name (or only a null entry), 0 is returned.
|
error_t envz_add (char **envz, size_t *envz_len, const char *name, const char *value) | Function |
The envz_add function adds an entry to *envz
(updating *envz and *envz_len ) with the name
name, and value value. If an entry with the same name
already exists in envz, it is removed first. If value is
0 , then the new entry will the special null type of entry
(mentioned above).
|
error_t envz_merge (char **envz, size_t *envz_len, const char *envz2, size_t envz2_len, int override) | Function |
The envz_merge function adds each entry in envz2 to envz,
as if with envz_add , updating *envz and
*envz_len . If override is true, then values in envz2
will supersede those with the same name in envz, otherwise not.
Null entries are treated just like other entries in this respect, so a null entry in envz can prevent an entry of the same name in envz2 from being added to envz, if override is false. |
void envz_strip (char **envz, size_t *envz_len) | Function |
The envz_strip function removes any null entries from envz,
updating *envz and *envz_len .
|
Character sets used in the early days of computing had only six, seven, or eight bits for each character: there was never a case where more than eight bits (one byte) were used to represent a single character. The limitations of this approach became more apparent as more people grappled with non-Roman character sets, where not all the characters that make up a language's character set can be represented by 2^8 choices. This chapter shows the functionality that was added to the C library to support multiple character sets.
A variety of solutions is available to overcome the differences between character sets with a 1:1 relation between bytes and characters and character sets with ratios of 2:1 or 4:1. The remainder of this section gives a few examples to help understand the design decisions made while developing the functionality of the C library.
A distinction we have to make right away is between internal and external representation. Internal representation means the representation used by a program while keeping the text in memory. External representations are used when text is stored or transmitted through some communication channel. Examples of external representations include files waiting in a directory to be read and parsed.
Traditionally there has been no difference between the two representations. It was equally comfortable and useful to use the same single-byte representation internally and externally. This comfort level decreases with more and larger character sets.
One of the problems to overcome with the internal representation is handling text that is externally encoded using different character sets. Assume a program that reads two texts and compares them using some metric. The comparison can be usefully done only if the texts are internally kept in a common format.
For such a common format (= character set) eight bits are certainly no longer enough. So the smallest entity will have to grow: wide characters will now be used. Instead of one byte per character, two or four will be used instead. (Three are not good to address in memory and more than four bytes seem not to be necessary).
As shown in some other part of this manual,
a completely new family has been created of functions that can handle wide
character texts in memory. The most commonly used character sets for such
internal wide character representations are Unicode and ISO 10646
(also known as UCS for Universal Character Set). Unicode was originally
planned as a 16-bit character set; whereas, ISO 10646 was designed to
be a 31-bit large code space. The two standards are practically identical.
They have the same character repertoire and code table, but Unicode specifies
added semantics. At the moment, only characters in the first 0x10000
code positions (the so-called Basic Multilingual Plane, BMP) have been
assigned, but the assignment of more specialized characters outside this
16-bit space is already in progress. A number of encodings have been
defined for Unicode and ISO 10646 characters:
UCS-2 is a 16-bit word that can only represent characters
from the BMP, UCS-4 is a 32-bit word than can represent any Unicode
and ISO 10646 character, UTF-8 is an ASCII compatible encoding where
ASCII characters are represented by ASCII bytes and non-ASCII characters
by sequences of 2-6 non-ASCII bytes, and finally UTF-16 is an extension
of UCS-2 in which pairs of certain UCS-2 words can be used to encode
non-BMP characters up to 0x10ffff
.
To represent wide characters the char
type is not suitable. For
this reason the ISO C standard introduces a new type that is
designed to keep one character of a wide character string. To maintain
the similarity there is also a type corresponding to int
for
those functions that take a single wide character.
wchar_t | Data type |
This data type is used as the base type for wide character strings.
In other words, arrays of objects of this type are the equivalent of
char[] for multibyte character strings. The type is defined in
stddef.h .
The ISO C90 standard, where But for GNU systems |
wint_t | Data type |
wint_t is a data type used for parameters and variables that
contain a single wide character. As the name suggests this type is the
equivalent of int when using the normal char strings. The
types wchar_t and wint_t often have the same
representation if their size is 32 bits wide but if wchar_t is
defined as char the type wint_t must be defined as
int due to the parameter promotion.
This type is defined in |
As there are for the char
data type macros are available for
specifying the minimum and maximum value representable in an object of
type wchar_t
.
wint_t WCHAR_MIN | Macro |
The macro WCHAR_MIN evaluates to the minimum value representable
by an object of type wint_t .
This macro was introduced in Amendment 1 to ISO C90. |
wint_t WCHAR_MAX | Macro |
The macro WCHAR_MAX evaluates to the maximum value representable
by an object of type wint_t .
This macro was introduced in Amendment 1 to ISO C90. |
Another special wide character value is the equivalent to EOF
.
wint_t WEOF | Macro |
The macro WEOF evaluates to a constant expression of type
wint_t whose value is different from any member of the extended
character set.
{ int c; ... while ((c = getc (fp)) < 0) ... } has to be rewritten to use { wint_t c; ... while ((c = wgetc (fp)) != WEOF) ... } This macro was introduced in Amendment 1 to ISO C90 and is
defined in |
These internal representations present problems when it comes to storing and transmittal. Because each single wide character consists of more than one byte, they are effected by byte-ordering. Thus, machines with different endianesses would see different values when accessing the same data. This byte ordering concern also applies for communication protocols that are all byte-based and, thereforet require that the sender has to decide about splitting the wide character in bytes. A last (but not least important) point is that wide characters often require more storage space than a customized byte-oriented character set.
For all the above reasons, an external encoding that is different from
the internal encoding is often used if the latter is UCS-2 or UCS-4.
The external encoding is byte-based and can be chosen appropriately for
the environment and for the texts to be handled. A variety of different
character sets can be used for this external encoding (information that
will not be exhaustively presented here-instead, a description of the
major groups will suffice). All of the ASCII-based character sets
fulfill one requirement: they are "filesystem safe." This means that
the character '/'
is used in the encoding only to
represent itself. Things are a bit different for character sets like
EBCDIC (Extended Binary Coded Decimal Interchange Code, a character set
family used by IBM), but if the operation system does not understand
EBCDIC directly the parameters-to-system calls have to be converted
first anyhow.
In most uses of ISO 2022 the defined character sets do not allow state changes that cover more than the next character. This has the big advantage that whenever one can identify the beginning of the byte sequence of a character one can interpret a text correctly. Examples of character sets using this policy are the various EUC character sets (used by Sun's operations systems, EUC-JP, EUC-KR, EUC-TW, and EUC-CN) or Shift_JIS (SJIS, a Japanese encoding).
But there are also character sets using a state that is valid for more than one character and has to be changed by another byte sequence. Examples for this are ISO-2022-JP, ISO-2022-KR, and ISO-2022-CN.
0xc2 0x61
(non-spacing acute accent, followed by lower-case `a') to get the "small
a with acute" character. To get the acute accent character on its own,
one has to write 0xc2 0x20
(the non-spacing acute followed by a
space).
Character sets like ISO 6937 are used in some embedded systems such as teletex.
There were a few other attempts to encode ISO 10646 such as UTF-7, but UTF-8 is today the only encoding that should be used. In fact, with any luck UTF-8 will soon be the only external encoding that has to be supported. It proves to be universally usable and its only disadvantage is that it favors Roman languages by making the byte string representation of other scripts (Cyrillic, Greek, Asian scripts) longer than necessary if using a specific character set for these scripts. Methods like the Unicode compression scheme can alleviate these problems.
The question remaining is: how to select the character set or encoding to use. The answer: you cannot decide about it yourself, it is decided by the developers of the system or the majority of the users. Since the goal is interoperability one has to use whatever the other people one works with use. If there are no constraints, the selection is based on the requirements the expected circle of users will have. In other words, if a project is expected to be used in only, say, Russia it is fine to use KOI8-R or a similar character set. But if at the same time people from, say, Greece are participating one should use a character set that allows all people to collaborate.
The most widely useful solution seems to be: go with the most general character set, namely ISO 10646. Use UTF-8 as the external encoding and problems about users not being able to use their own language adequately are a thing of the past.
One final comment about the choice of the wide character representation
is necessary at this point. We have said above that the natural choice
is using Unicode or ISO 10646. This is not required, but at least
encouraged, by the ISO C standard. The standard defines at least a
macro __STDC_ISO_10646__
that is only defined on systems where
the wchar_t
type encodes ISO 10646 characters. If this
symbol is not defined one should avoid making assumptions about the wide
character representation. If the programmer uses only the functions
provided by the C library to handle wide character strings there should
be no compatibility problems with other systems.
A Unix C library contains three different sets of functions in two families to handle character set conversion. One of the function families (the most commonly used) is specified in the ISO C90 standard and, therefore, is portable even beyond the Unix world. Unfortunately this family is the least useful one. These functions should be avoided whenever possible, especially when developing libraries (as opposed to applications).
The second family of functions got introduced in the early Unix standards (XPG2) and is still part of the latest and greatest Unix standard: Unix 98. It is also the most powerful and useful set of functions. But we will start with the functions defined in Amendment 1 to ISO C90.
The ISO C standard defines functions to convert strings from a multibyte representation to wide character strings. There are a number of peculiarities:
LC_CTYPE
category of the current locale is used; see
Locale Categories.
Despite these limitations the ISO C functions can be used in many
contexts. In graphical user interfaces, for instance, it is not
uncommon to have functions that require text to be displayed in a wide
character string if the text is not simple ASCII. The text itself might
come from a file with translations and the user should decide about the
current locale, which determines the translation and therefore also the
external encoding used. In such a situation (and many others) the
functions described here are perfect. If more freedom while performing
the conversion is necessary take a look at the iconv
functions
(see Generic Charset Conversion).
We already said above that the currently selected locale for the
LC_CTYPE
category decides about the conversion that is performed
by the functions we are about to describe. Each locale uses its own
character set (given as an argument to localedef
) and this is the
one assumed as the external multibyte encoding. The wide character
character set always is UCS-4, at least on GNU systems.
A characteristic of each multibyte character set is the maximum number of bytes that can be necessary to represent one character. This information is quite important when writing code that uses the conversion functions (as shown in the examples below). The ISO C standard defines two macros that provide this information.
int MB_LEN_MAX | Macro |
MB_LEN_MAX specifies the maximum number of bytes in the multibyte
sequence for a single character in any of the supported locales. It is
a compile-time constant and is defined in limits.h .
|
int MB_CUR_MAX | Macro |
MB_CUR_MAX expands into a positive integer expression that is the
maximum number of bytes in a multibyte character in the current locale.
The value is never greater than MB_LEN_MAX . Unlike
MB_LEN_MAX this macro need not be a compile-time constant, and in
the GNU C library it is not.
|
Two different macros are necessary since strictly ISO C90 compilers
do not allow variable length array definitions, but still it is desirable
to avoid dynamic allocation. This incomplete piece of code shows the
problem:
{ char buf[MB_LEN_MAX]; ssize_t len = 0; while (! feof (fp)) { fread (&buf[len], 1, MB_CUR_MAX - len, fp); /* ... process buf */ len -= used; } }
The code in the inner loop is expected to have always enough bytes in
the array buf to convert one multibyte character. The array
buf has to be sized statically since many compilers do not allow a
variable size. The fread
call makes sure that MB_CUR_MAX
bytes are always available in buf. Note that it isn't
a problem if MB_CUR_MAX
is not a compile-time constant.
In the introduction of this chapter it was said that certain character sets use a stateful encoding. That is, the encoded values depend in some way on the previous bytes in the text.
Since the conversion functions allow converting a text in more than one step we must have a way to pass this information from one call of the functions to another.
mbstate_t | Data type |
A variable of type mbstate_t can contain all the information
about the shift state needed from one call to a conversion
function to another.
|
To use objects of type mbstate_t
the programmer has to define such
objects (normally as local variables on the stack) and pass a pointer to
the object to the conversion functions. This way the conversion function
can update the object if the current multibyte character set is stateful.
There is no specific function or initializer to put the state object in
any specific state. The rules are that the object should always
represent the initial state before the first use, and this is achieved by
clearing the whole variable with code such as follows:
{ mbstate_t state; memset (&state, '\0', sizeof (state)); /* from now on state can be used. */ ... }
When using the conversion functions to generate output it is often necessary to test whether the current state corresponds to the initial state. This is necessary, for example, to decide whether to emit escape sequences to set the state to the initial state at certain sequence points. Communication protocols often require this.
int mbsinit (const mbstate_t *ps) | Function |
The mbsinit function determines whether the state object pointed
to by ps is in the initial state. If ps is a null pointer or
the object is in the initial state the return value is nonzero. Otherwise
it is zero.
|
Code using mbsinit
often looks similar to this:
{ mbstate_t state; memset (&state, '\0', sizeof (state)); /* Use state. */ ... if (! mbsinit (&state)) { /* Emit code to return to initial state. */ const wchar_t empty[] = L""; const wchar_t *srcp = empty; wcsrtombs (outbuf, &srcp, outbuflen, &state); } ... }
The code to emit the escape sequence to get back to the initial state is
interesting. The wcsrtombs
function can be used to determine the
necessary output code (see Converting Strings). Please note that on
GNU systems it is not necessary to perform this extra action for the
conversion from multibyte text to wide character text since the wide
character encoding is not stateful. But there is nothing mentioned in
any standard that prohibits making wchar_t
using a stateful
encoding.
The most fundamental of the conversion functions are those dealing with single characters. Please note that this does not always mean single bytes. But since there is very often a subset of the multibyte character set that consists of single byte sequences, there are functions to help with converting bytes. Frequently, ASCII is a subpart of the multibyte character set. In such a scenario, each ASCII character stands for itself, and all other characters have at least a first byte that is beyond the range 0 to 127.
wint_t btowc (int c) | Function |
The btowc function ("byte to wide character") converts a valid
single byte character c in the initial shift state into the wide
character equivalent using the conversion rules from the currently
selected locale of the LC_CTYPE category.
If Please note the restriction of c being tested for validity only in
the initial shift state. No The |
Despite the limitation that the single byte value always is interpreted
in the initial state this function is actually useful most of the time.
Most characters are either entirely single-byte character sets or they
are extension to ASCII. But then it is possible to write code like this
(not that this specific example is very useful):
wchar_t * itow (unsigned long int val) { static wchar_t buf[30]; wchar_t *wcp = &buf[29]; *wcp = L'\0'; while (val != 0) { *--wcp = btowc ('0' + val % 10); val /= 10; } if (wcp == &buf[29]) *--wcp = L'0'; return wcp; }
Why is it necessary to use such a complicated implementation and not
simply cast '0' + val % 10
to a wide character? The answer is
that there is no guarantee that one can perform this kind of arithmetic
on the character of the character set used for wchar_t
representation. In other situations the bytes are not constant at
compile time and so the compiler cannot do the work. In situations like
this it is necessary btowc
.
There also is a function for the conversion in the other direction.
int wctob (wint_t c) | Function |
The wctob function ("wide character to byte") takes as the
parameter a valid wide character. If the multibyte representation for
this character in the initial state is exactly one byte long, the return
value of this function is this character. Otherwise the return value is
EOF .
|
There are more general functions to convert single character from multibyte representation to wide characters and vice versa. These functions pose no limit on the length of the multibyte representation and they also do not require it to be in the initial state.
size_t mbrtowc (wchar_t *restrict pwc, const char *restrict s, size_t n, mbstate_t *restrict ps) | Function |
The mbrtowc function ("multibyte restartable to wide
character") converts the next multibyte character in the string pointed
to by s into a wide character and stores it in the wide character
string pointed to by pwc. The conversion is performed according
to the locale currently selected for the LC_CTYPE category. If
the conversion for the character set used in the locale requires a state,
the multibyte string is interpreted in the state represented by the
object pointed to by ps. If ps is a null pointer, a static,
internal state variable used only by the mbrtowc function is
used.
If the next multibyte character corresponds to the NUL wide character,
the return value of the function is 0 and the state object is
afterwards in the initial state. If the next n or fewer bytes
form a correct multibyte character, the return value is the number of
bytes starting from s that form the multibyte character. The
conversion state is updated according to the bytes consumed in the
conversion. In both cases the wide character (either the If the first n bytes of the multibyte string possibly form a valid
multibyte character but there are more than n bytes needed to
complete it, the return value of the function is If the first
|
Use of mbrtowc
is straightforward. A function that copies a
multibyte string into a wide character string while at the same time
converting all lowercase characters into uppercase could look like this
(this is not the final version, just an example; it has no error
checking, and sometimes leaks memory):
wchar_t * mbstouwcs (const char *s) { size_t len = strlen (s); wchar_t *result = malloc ((len + 1) * sizeof (wchar_t)); wchar_t *wcp = result; wchar_t tmp[1]; mbstate_t state; size_t nbytes; memset (&state, '\0', sizeof (state)); while ((nbytes = mbrtowc (tmp, s, len, &state)) > 0) { if (nbytes >= (size_t) -2) /* Invalid input string. */ return NULL; *result++ = towupper (tmp[0]); len -= nbytes; s += nbytes; } return result; }
The use of mbrtowc
should be clear. A single wide character is
stored in tmp[0]
, and the number of consumed bytes is stored
in the variable nbytes. If the conversion is successful, the
uppercase variant of the wide character is stored in the result
array and the pointer to the input string and the number of available
bytes is adjusted.
The only non-obvious thing about mbrtowc
might be the way memory
is allocated for the result. The above code uses the fact that there
can never be more wide characters in the converted results than there are
bytes in the multibyte input string. This method yields a pessimistic
guess about the size of the result, and if many wide character strings
have to be constructed this way or if the strings are long, the extra
memory required to be allocated because the input string contains
multibyte characters might be significant. The allocated memory block can
be resized to the correct size before returning it, but a better solution
might be to allocate just the right amount of space for the result right
away. Unfortunately there is no function to compute the length of the wide
character string directly from the multibyte string. There is, however, a
function that does part of the work.
size_t mbrlen (const char *restrict s, size_t n, mbstate_t *ps) | Function |
The mbrlen function ("multibyte restartable length") computes
the number of at most n bytes starting at s, which form the
next valid and complete multibyte character.
If the next multibyte character corresponds to the NUL wide character, the return value is 0. If the next n bytes form a valid multibyte character, the number of bytes belonging to this multibyte character byte sequence is returned. If the the first n bytes possibly form a valid multibyte
character but the character is incomplete, the return value is
The multibyte sequence is interpreted in the state represented by the
object pointed to by ps. If ps is a null pointer, a state
object local to
|
The attentive reader now will note that mbrlen
can be implemented
as
mbrtowc (NULL, s, n, ps != NULL ? ps : &internal)
This is true and in fact is mentioned in the official specification.
How can this function be used to determine the length of the wide
character string created from a multibyte character string? It is not
directly usable, but we can define a function mbslen
using it:
size_t mbslen (const char *s) { mbstate_t state; size_t result = 0; size_t nbytes; memset (&state, '\0', sizeof (state)); while ((nbytes = mbrlen (s, MB_LEN_MAX, &state)) > 0) { if (nbytes >= (size_t) -2) /* Something is wrong. */ return (size_t) -1; s += nbytes; ++result; } return result; }
This function simply calls mbrlen
for each multibyte character
in the string and counts the number of function calls. Please note that
we here use MB_LEN_MAX
as the size argument in the mbrlen
call. This is acceptable since a) this value is larger then the length of
the longest multibyte character sequence and b) we know that the string
s ends with a NUL byte, which cannot be part of any other multibyte
character sequence but the one representing the NUL wide character.
Therefore, the mbrlen
function will never read invalid memory.
Now that this function is available (just to make this clear, this
function is not part of the GNU C library) we can compute the
number of wide character required to store the converted multibyte
character string s using
wcs_bytes = (mbslen (s) + 1) * sizeof (wchar_t);
Please note that the mbslen
function is quite inefficient. The
implementation of mbstouwcs
with mbslen
would have to
perform the conversion of the multibyte character input string twice, and
this conversion might be quite expensive. So it is necessary to think
about the consequences of using the easier but imprecise method before
doing the work twice.
size_t wcrtomb (char *restrict s, wchar_t wc, mbstate_t *restrict ps) | Function |
The wcrtomb function ("wide character restartable to
multibyte") converts a single wide character into a multibyte string
corresponding to that wide character.
If s is a null pointer, the function resets the state stored in
the objects pointed to by ps (or the internal wcrtombs (temp_buf, L'\0', ps) since, if s is a null pointer, If wc is the NUL wide character, Otherwise a byte sequence (possibly including shift sequences) is written
into the string s. This only happens if wc is a valid wide
character (i.e., it has a multibyte representation in the character set
selected by locale of the If no error occurred the function returns the number of bytes stored in the string s. This includes all bytes representing shift sequences. One word about the interface of the function: there is no parameter
specifying the length of the array s. Instead the function
assumes that there are at least
|
Using wcrtomb
is as easy as using mbrtowc
. The following
example appends a wide character string to a multibyte character string.
Again, the code is not really useful (or correct), it is simply here to
demonstrate the use and some problems.
char * mbscatwcs (char *s, size_t len, const wchar_t *ws) { mbstate_t state; /* Find the end of the existing string. */ char *wp = strchr (s, '\0'); len -= wp - s; memset (&state, '\0', sizeof (state)); do { size_t nbytes; if (len < MB_CUR_LEN) { /* We cannot guarantee that the next character fits into the buffer, so return an error. */ errno = E2BIG; return NULL; } nbytes = wcrtomb (wp, *ws, &state); if (nbytes == (size_t) -1) /* Error in the conversion. */ return NULL; len -= nbytes; wp += nbytes; } while (*ws++ != L'\0'); return s; }
First the function has to find the end of the string currently in the
array s. The strchr
call does this very efficiently since a
requirement for multibyte character representations is that the NUL byte
is never used except to represent itself (and in this context, the end
of the string).
After initializing the state object the loop is entered where the first
task is to make sure there is enough room in the array s. We
abort if there are not at least MB_CUR_LEN
bytes available. This
is not always optimal but we have no other choice. We might have less
than MB_CUR_LEN
bytes available but the next multibyte character
might also be only one byte long. At the time the wcrtomb
call
returns it is too late to decide whether the buffer was large enough. If
this solution is unsuitable, there is a very slow but more accurate
solution.
... if (len < MB_CUR_LEN) { mbstate_t temp_state; memcpy (&temp_state, &state, sizeof (state)); if (wcrtomb (NULL, *ws, &temp_state) > len) { /* We cannot guarantee that the next character fits into the buffer, so return an error. */ errno = E2BIG; return NULL; } } ...
Here we perform the conversion that might overflow the buffer so that
we are afterwards in the position to make an exact decision about the
buffer size. Please note the NULL
argument for the destination
buffer in the new wcrtomb
call; since we are not interested in the
converted text at this point, this is a nice way to express this. The
most unusual thing about this piece of code certainly is the duplication
of the conversion state object, but if a change of the state is necessary
to emit the next multibyte character, we want to have the same shift state
change performed in the real conversion. Therefore, we have to preserve
the initial shift state information.
There are certainly many more and even better solutions to this problem. This example is only provided for educational purposes.
The functions described in the previous section only convert a single character at a time. Most operations to be performed in real-world programs include strings and therefore the ISO C standard also defines conversions on entire strings. However, the defined set of functions is quite limited; therefore, the GNU C library contains a few extensions that can help in some important situations.
size_t mbsrtowcs (wchar_t *restrict dst, const char **restrict src, size_t len, mbstate_t *restrict ps) | Function |
The mbsrtowcs function ("multibyte string restartable to wide
character string") converts an NUL-terminated multibyte character
string at *src into an equivalent wide character string,
including the NUL wide character at the end. The conversion is started
using the state information from the object pointed to by ps or
from an internal object of mbsrtowcs if ps is a null
pointer. Before returning, the state object is updated to match the state
after the last converted character. The state is the initial state if the
terminating NUL byte is reached and converted.
If dst is not a null pointer, the result is stored in the array pointed to by dst; otherwise, the conversion result is not available since it is stored in an internal buffer. If len wide characters are stored in the array dst before reaching the end of the input string, the conversion stops and len is returned. If dst is a null pointer, len is never checked. Another reason for a premature return from the function call is if the
input string contains an invalid multibyte sequence. In this case the
global variable In all other cases the function returns the number of wide characters
converted during this call. If dst is not null,
|
The definition of the mbsrtowcs
function has one important
limitation. The requirement that dst has to be a NUL-terminated
string provides problems if one wants to convert buffers with text. A
buffer is normally no collection of NUL-terminated strings but instead a
continuous collection of lines, separated by newline characters. Now
assume that a function to convert one line from a buffer is needed. Since
the line is not NUL-terminated, the source pointer cannot directly point
into the unmodified text buffer. This means, either one inserts the NUL
byte at the appropriate place for the time of the mbsrtowcs
function call (which is not doable for a read-only buffer or in a
multi-threaded application) or one copies the line in an extra buffer
where it can be terminated by a NUL byte. Note that it is not in general
possible to limit the number of characters to convert by setting the
parameter len to any specific value. Since it is not known how
many bytes each multibyte character sequence is in length, one can only
guess.
There is still a problem with the method of NUL-terminating a line right
after the newline character, which could lead to very strange results.
As said in the description of the mbsrtowcs
function above the
conversion state is guaranteed to be in the initial shift state after
processing the NUL byte at the end of the input string. But this NUL
byte is not really part of the text (i.e., the conversion state after
the newline in the original text could be something different than the
initial shift state and therefore the first character of the next line
is encoded using this state). But the state in question is never
accessible to the user since the conversion stops after the NUL byte
(which resets the state). Most stateful character sets in use today
require that the shift state after a newline be the initial state-but
this is not a strict guarantee. Therefore, simply NUL-terminating a
piece of a running text is not always an adequate solution and,
therefore, should never be used in generally used code.
The generic conversion interface (see Generic Charset Conversion)
does not have this limitation (it simply works on buffers, not
strings), and the GNU C library contains a set of functions that take
additional parameters specifying the maximal number of bytes that are
consumed from the input string. This way the problem of
mbsrtowcs
's example above could be solved by determining the line
length and passing this length to the function.
size_t wcsrtombs (char *restrict dst, const wchar_t **restrict src, size_t len, mbstate_t *restrict ps) | Function |
The wcsrtombs function ("wide character string restartable to
multibyte string") converts the NUL-terminated wide character string at
*src into an equivalent multibyte character string and
stores the result in the array pointed to by dst. The NUL wide
character is also converted. The conversion starts in the state
described in the object pointed to by ps or by a state object
locally to wcsrtombs in case ps is a null pointer. If
dst is a null pointer, the conversion is performed as usual but the
result is not available. If all characters of the input string were
successfully converted and if dst is not a null pointer, the
pointer pointed to by src gets assigned a null pointer.
If one of the wide characters in the input string has no valid multibyte
character equivalent, the conversion stops early, sets the global
variable Another reason for a premature stop is if dst is not a null pointer and the next converted character would require more than len bytes in total to the array dst. In this case (and if dest is not a null pointer) the pointer pointed to by src is assigned a value pointing to the wide character right after the last one successfully converted. Except in the case of an encoding error the return value of the
The |
The restriction mentioned above for the mbsrtowcs
function applies
here also. There is no possibility of directly controlling the number of
input characters. One has to place the NUL wide character at the correct
place or control the consumed input indirectly via the available output
array size (the len parameter).
size_t mbsnrtowcs (wchar_t *restrict dst, const char **restrict src, size_t nmc, size_t len, mbstate_t *restrict ps) | Function |
The mbsnrtowcs function is very similar to the mbsrtowcs
function. All the parameters are the same except for nmc, which is
new. The return value is the same as for mbsrtowcs .
This new parameter specifies how many bytes at most can be used from the
multibyte character string. In other words, the multibyte character
string This function is a GNU extension. It is meant to work around the problems mentioned above. Now it is possible to convert a buffer with multibyte character text piece for piece without having to care about inserting NUL bytes and the effect of NUL bytes on the conversion state. |
A function to convert a multibyte string into a wide character string
and display it could be written like this (this is not a really useful
example):
void showmbs (const char *src, FILE *fp) { mbstate_t state; int cnt = 0; memset (&state, '\0', sizeof (state)); while (1) { wchar_t linebuf[100]; const char *endp = strchr (src, '\n'); size_t n; /* Exit if there is no more line. */ if (endp == NULL) break; n = mbsnrtowcs (linebuf, &src, endp - src, 99, &state); linebuf[n] = L'\0'; fprintf (fp, "line %d: \"%S\"\n", linebuf); } }
There is no problem with the state after a call to mbsnrtowcs
.
Since we don't insert characters in the strings that were not in there
right from the beginning and we use state only for the conversion
of the given buffer, there is no problem with altering the state.
size_t wcsnrtombs (char *restrict dst, const wchar_t **restrict src, size_t nwc, size_t len, mbstate_t *restrict ps) | Function |
The wcsnrtombs function implements the conversion from wide
character strings to multibyte character strings. It is similar to
wcsrtombs but, just like mbsnrtowcs , it takes an extra
parameter, which specifies the length of the input string.
No more than nwc wide characters from the input string
The |
The example programs given in the last sections are only brief and do
not contain all the error checking, etc. Presented here is a complete
and documented example. It features the mbrtowc
function but it
should be easy to derive versions using the other functions.
int file_mbsrtowcs (int input, int output) { /* Note the use ofMB_LEN_MAX
.MB_CUR_MAX
cannot portably be used here. */ char buffer[BUFSIZ + MB_LEN_MAX]; mbstate_t state; int filled = 0; int eof = 0; /* Initialize the state. */ memset (&state, '\0', sizeof (state)); while (!eof) { ssize_t nread; ssize_t nwrite; char *inp = buffer; wchar_t outbuf[BUFSIZ]; wchar_t *outp = outbuf; /* Fill up the buffer from the input file. */ nread = read (input, buffer + filled, BUFSIZ); if (nread < 0) { perror ("read"); return 0; } /* If we reach end of file, make a note to read no more. */ if (nread == 0) eof = 1; /*filled
is now the number of bytes inbuffer
. */ filled += nread; /* Convert those bytes to wide characters-as many as we can. */ while (1) { size_t thislen = mbrtowc (outp, inp, filled, &state); /* Stop converting at invalid character; this can mean we have read just the first part of a valid character. */ if (thislen == (size_t) -1) break; /* We want to handle embedded NUL bytes but the return value is 0. Correct this. */ if (thislen == 0) thislen = 1; /* Advance past this character. */ inp += thislen; filled -= thislen; ++outp; } /* Write the wide characters we just made. */ nwrite = write (output, outbuf, (outp - outbuf) * sizeof (wchar_t)); if (nwrite < 0) { perror ("write"); return 0; } /* See if we have a real invalid character. */ if ((eof && filled > 0) || filled >= MB_CUR_MAX) { error (0, 0, "invalid multibyte character"); return 0; } /* If any characters must be carried forward, put them at the beginning ofbuffer
. */ if (filled > 0) memmove (inp, buffer, filled); } return 1; }
The functions described in the previous chapter are defined in Amendment 1 to ISO C90, but the original ISO C90 standard also contained functions for character set conversion. The reason that these original functions are not described first is that they are almost entirely useless.
The problem is that all the conversion functions described in the original ISO C90 use a local state. Using a local state implies that multiple conversions at the same time (not only when using threads) cannot be done, and that you cannot first convert single characters and then strings since you cannot tell the conversion functions which state to use.
These original functions are therefore usable only in a very limited set of situations. One must complete converting the entire string before starting a new one, and each string/text must be converted with the same function (there is no problem with the library itself; it is guaranteed that no library function changes the state of any of these functions). For the above reasons it is highly requested that the functions described in the previous section be used in place of non-reentrant conversion functions.
int mbtowc (wchar_t *restrict result, const char *restrict string, size_t size) | Function |
The mbtowc ("multibyte to wide character") function when called
with non-null string converts the first multibyte character
beginning at string to its corresponding wide character code. It
stores the result in *result .
For a valid multibyte character, For an invalid byte sequence, If the multibyte character code uses shift characters, then
|
int wctomb (char *string, wchar_t wchar) | Function |
The wctomb ("wide character to multibyte") function converts
the wide character code wchar to its corresponding multibyte
character sequence, and stores the result in bytes starting at
string. At most MB_CUR_MAX characters are stored.
Given a valid code, If wchar is an invalid wide character code, If the multibyte character code uses shift characters, then
Calling this function with a wchar argument of zero when
string is not null has the side-effect of reinitializing the
stored shift state as well as storing the multibyte character
|
Similar to mbrlen
there is also a non-reentrant function that
computes the length of a multibyte character. It can be defined in
terms of mbtowc
.
int mblen (const char *string, size_t size) | Function |
The mblen function with a non-null string argument returns
the number of bytes that make up the multibyte character beginning at
string, never examining more than size bytes. (The idea is
to supply for size the number of bytes of data you have in hand.)
The return value of For a valid multibyte character, If the multibyte character code uses shift characters, then The function |
For convenience the ISO C90 standard also defines functions to convert entire strings instead of single characters. These functions suffer from the same problems as their reentrant counterparts from Amendment 1 to ISO C90; see Converting Strings.
size_t mbstowcs (wchar_t *wstring, const char *string, size_t size) | Function |
The mbstowcs ("multibyte string to wide character string")
function converts the null-terminated string of multibyte characters
string to an array of wide character codes, storing not more than
size wide characters into the array beginning at wstring.
The terminating null character counts towards the size, so if size
is less than the actual number of wide characters resulting from
string, no terminating null character is stored.
The conversion of characters from string begins in the initial shift state. If an invalid multibyte character sequence is found, the Here is an example showing how to convert a string of multibyte
characters, allocating enough space for the result.
wchar_t * mbstowcs_alloc (const char *string) { size_t size = strlen (string) + 1; wchar_t *buf = xmalloc (size * sizeof (wchar_t)); size = mbstowcs (buf, string, size); if (size == (size_t) -1) return NULL; buf = xrealloc (buf, (size + 1) * sizeof (wchar_t)); return buf; } |
size_t wcstombs (char *string, const wchar_t *wstring, size_t size) | Function |
The wcstombs ("wide character string to multibyte string")
function converts the null-terminated wide character array wstring
into a string containing multibyte characters, storing not more than
size bytes starting at string, followed by a terminating
null character if there is room. The conversion of characters begins in
the initial shift state.
The terminating null character counts towards the size, so if size is less than or equal to the number of bytes needed in wstring, no terminating null character is stored. If a code that does not correspond to a valid multibyte character is
found, the |
In some multibyte character codes, the meaning of any particular byte sequence is not fixed; it depends on what other sequences have come earlier in the same string. Typically there are just a few sequences that can change the meaning of other sequences; these few are called shift sequences and we say that they set the shift state for other sequences that follow.
To illustrate shift state and shift sequences, suppose we decide that
the sequence 0200
(just one byte) enters Japanese mode, in which
pairs of bytes in the range from 0240
to 0377
are single
characters, while 0201
enters Latin-1 mode, in which single bytes
in the range from 0240
to 0377
are characters, and
interpreted according to the ISO Latin-1 character set. This is a
multibyte code that has two alternative shift states ("Japanese mode"
and "Latin-1 mode"), and two shift sequences that specify particular
shift states.
When the multibyte character code in use has shift states, then
mblen
, mbtowc
, and wctomb
must maintain and update
the current shift state as they scan the string. To make this work
properly, you must follow these rules:
mblen (NULL,
0)
. This initializes the shift state to its standard initial value.
Here is an example of using mblen
following these rules:
void scan_string (char *s) { int length = strlen (s); /* Initialize shift state. */ mblen (NULL, 0); while (1) { int thischar = mblen (s, length); /* Deal with end of string and invalid characters. */ if (thischar == 0) break; if (thischar == -1) { error ("invalid multibyte character"); break; } /* Advance past this character. */ s += thischar; length -= thischar; } }
The functions mblen
, mbtowc
and wctomb
are not
reentrant when using a multibyte code that uses a shift state. However,
no other library functions call these functions, so you don't have to
worry that the shift state will be changed mysteriously.
The conversion functions mentioned so far in this chapter all had in
common that they operate on character sets that are not directly
specified by the functions. The multibyte encoding used is specified by
the currently selected locale for the LC_CTYPE
category. The
wide character set is fixed by the implementation (in the case of GNU C
library it is always UCS-4 encoded ISO 10646.
This has of course several problems when it comes to general character conversion:
LC_CTYPE
category, one has to change the LC_CTYPE
locale using
setlocale
.
Changing the LC_TYPE
locale introduces major problems for the rest
of the programs since several more functions (e.g., the character
classification functions, see Classification of Characters) use the
LC_CTYPE
category.
LC_CTYPE
selection is global and shared by all
threads.
wchar_t
representation, there is at least a two-step
process necessary to convert a text using the functions above. One would
have to select the source character set as the multibyte encoding,
convert the text into a wchar_t
text, select the destination
character set as the multibyte encoding, and convert the wide character
text to the multibyte (= destination) character set.
Even if this is possible (which is not guaranteed) it is a very tiring work. Plus it suffers from the other two raised points even more due to the steady changing of the locale.
The XPG2 standard defines a completely new set of functions, which has none of these limitations. They are not at all coupled to the selected locales, and they have no constraints on the character sets selected for source and destination. Only the set of available conversions limits them. The standard does not specify that any conversion at all must be available. Such availability is a measure of the quality of the implementation.
In the following text first the interface to iconv
and then the
conversion function, will be described. Comparisons with other
implementations will show what obstacles stand in the way of portable
applications. Finally, the implementation is described in so far as might
interest the advanced user who wants to extend conversion capabilities.
iconv
example.
iconv
Implementations.
iconv
Implementation in the GNU C
library.
This set of functions follows the traditional cycle of using a resource: open-use-close. The interface consists of three functions, each of which implements one step.
Before the interfaces are described it is necessary to introduce a
data type. Just like other open-use-close interfaces the functions
introduced here work using handles and the iconv.h
header
defines a special type for the handles used.
iconv_t | Data Type |
This data type is an abstract type defined in iconv.h . The user
must not assume anything about the definition of this type; it must be
completely opaque.
Objects of this type can get assigned handles for the conversions using
the |
The first step is the function to create a handle.
iconv_t iconv_open (const char *tocode, const char *fromcode) | Function |
The iconv_open function has to be used before starting a
conversion. The two parameters this function takes determine the
source and destination character set for the conversion, and if the
implementation has the possibility to perform such a conversion, the
function returns a handle.
If the wanted conversion is not available, the
It is not possible to use the same descriptor in different threads to perform independent conversions. The data structures associated with the descriptor include information about the conversion state. This must not be messed up by using it in different conversions. An The GNU C library implementation of The |
The iconv
implementation can associate large data structure with
the handle returned by iconv_open
. Therefore, it is crucial to
free all the resources once all conversions are carried out and the
conversion is not needed anymore.
int iconv_close (iconv_t cd) | Function |
The iconv_close function frees all resources associated with the
handle cd, which must have been returned by a successful call to
the iconv_open function.
If the function call was successful the return value is 0.
Otherwise it is -1 and
The |
The standard defines only one actual conversion function. This has, therefore, the most general interface: it allows conversion from one buffer to another. Conversion from a file to a buffer, vice versa, or even file to file can be implemented on top of it.
size_t iconv (iconv_t cd, char **inbuf, size_t *inbytesleft, char **outbuf, size_t *outbytesleft) | Function |
The iconv function converts the text in the input buffer
according to the rules associated with the descriptor cd and
stores the result in the output buffer. It is possible to call the
function for the same text several times in a row since for stateful
character sets the necessary state information is kept in the data
structures associated with the descriptor.
The input buffer is specified by The output buffer is specified in a similar way. If inbuf is a null pointer, the The conversion stops for one of three reasons. The first is that all characters from the input buffer are converted. This actually can mean two things: either all bytes from the input buffer are consumed or there are some bytes at the end of the buffer that possibly can form a complete character but the input is incomplete. The second reason for a stop is that the output buffer is full. And the third reason is that the input contains invalid characters. In all of these cases the buffer pointers after the last successful conversion, for input and output buffer, are stored in inbuf and outbuf, and the available room in each buffer is stored in inbytesleft and outbytesleft. Since the character sets selected in the If all input from the input buffer is successfully converted and stored
in the output buffer, the function returns the number of non-reversible
conversions performed. In all other cases the return value is
The |
The definition of the iconv
function is quite good overall. It
provides quite flexible functionality. The only problems lie in the
boundary cases, which are incomplete byte sequences at the end of the
input buffer and invalid input. A third problem, which is not really
a design problem, is the way conversions are selected. The standard
does not say anything about the legitimate names, a minimal set of
available conversions. We will see how this negatively impacts other
implementations, as demonstrated below.
iconv
exampleThe example below features a solution for a common problem. Given that
one knows the internal encoding used by the system for wchar_t
strings, one often is in the position to read text from a file and store
it in wide character buffers. One can do this using mbsrtowcs
,
but then we run into the problems discussed above.
int
file2wcs (int fd, const char *charset, wchar_t *outbuf, size_t avail)
{
char inbuf[BUFSIZ];
size_t insize = 0;
char *wrptr = (char *) outbuf;
int result = 0;
iconv_t cd;
cd = iconv_open ("WCHAR_T", charset);
if (cd == (iconv_t) -1)
{
/* Something went wrong. */
if (errno == EINVAL)
error (0, 0, "conversion from '%s' to wchar_t not available",
charset);
else
perror ("iconv_open");
/* Terminate the output string. */
*outbuf = L'\0';
return -1;
}
while (avail > 0)
{
size_t nread;
size_t nconv;
char *inptr = inbuf;
/* Read more input. */
nread = read (fd, inbuf + insize, sizeof (inbuf) - insize);
if (nread == 0)
{
/* When we come here the file is completely read.
This still could mean there are some unused
characters in the inbuf
. Put them back. */
if (lseek (fd, -insize, SEEK_CUR) == -1)
result = -1;
/* Now write out the byte sequence to get into the
initial state if this is necessary. */
iconv (cd, NULL, NULL, &wrptr, &avail);
break;
}
insize += nread;
/* Do the conversion. */
nconv = iconv (cd, &inptr, &insize, &wrptr, &avail);
if (nconv == (size_t) -1)
{
/* Not everything went right. It might only be
an unfinished byte sequence at the end of the
buffer. Or it is a real problem. */
if (errno == EINVAL)
/* This is harmless. Simply move the unused
bytes to the beginning of the buffer so that
they can be used in the next round. */
memmove (inbuf, inptr, insize);
else
{
/* It is a real problem. Maybe we ran out of
space in the output buffer or we have invalid
input. In any case back the file pointer to
the position of the last processed byte. */
lseek (fd, -insize, SEEK_CUR);
result = -1;
break;
}
}
}
/* Terminate the output string. */
if (avail >= sizeof (wchar_t))
*((wchar_t *) wrptr) = L'\0';
if (iconv_close (cd) != 0)
perror ("iconv_close");
return (wchar_t *) wrptr - outbuf;
}
This example shows the most important aspects of using the iconv
functions. It shows how successive calls to iconv
can be used to
convert large amounts of text. The user does not have to care about
stateful encodings as the functions take care of everything.
An interesting point is the case where iconv
returns an error and
errno
is set to EINVAL
. This is not really an error in the
transformation. It can happen whenever the input character set contains
byte sequences of more than one byte for some character and texts are not
processed in one piece. In this case there is a chance that a multibyte
sequence is cut. The caller can then simply read the remainder of the
takes and feed the offending bytes together with new character from the
input to iconv
and continue the work. The internal state kept in
the descriptor is not unspecified after such an event as is the
case with the conversion functions from the ISO C standard.
The example also shows the problem of using wide character strings with
iconv
. As explained in the description of the iconv
function above, the function always takes a pointer to a char
array and the available space is measured in bytes. In the example, the
output buffer is a wide character buffer; therefore, we use a local
variable wrptr of type char *
, which is used in the
iconv
calls.
This looks rather innocent but can lead to problems on platforms that
have tight restriction on alignment. Therefore the caller of iconv
has to make sure that the pointers passed are suitable for access of
characters from the appropriate character set. Since, in the
above case, the input parameter to the function is a wchar_t
pointer, this is the case (unless the user violates alignment when
computing the parameter). But in other situations, especially when
writing generic functions where one does not know what type of character
set one uses and, therefore, treats text as a sequence of bytes, it might
become tricky.
iconv
ImplementationsThis is not really the place to discuss the iconv
implementation
of other systems but it is necessary to know a bit about them to write
portable programs. The above mentioned problems with the specification
of the iconv
functions can lead to portability issues.
The first thing to notice is that, due to the large number of character sets in use, it is certainly not practical to encode the conversions directly in the C library. Therefore, the conversion information must come from files outside the C library. This is usually done in one or both of the following ways:
This solution is problematic as it requires a great deal of effort to apply to all character sets (potentially an infinite set). The differences in the structure of the different character sets is so large that many different variants of the table-processing functions must be developed. In addition, the generic nature of these functions make them slower than specifically implemented functions.
This solution provides much more flexibility. The C library itself contains only very little code and therefore reduces the general memory footprint. Also, with a documented interface between the C library and the loadable modules it is possible for third parties to extend the set of available conversion modules. A drawback of this solution is that dynamic loading must be available.
Some implementations in commercial Unices implement a mixture of these possibilities; the majority implement only the second solution. Using loadable modules moves the code out of the library itself and keeps the door open for extensions and improvements, but this design is also limiting on some platforms since not many platforms support dynamic loading in statically linked programs. On platforms without this capability it is therefore not possible to use this interface in statically linked programs. The GNU C library has, on ELF platforms, no problems with dynamic loading in these situations; therefore, this point is moot. The danger is that one gets acquainted with this situation and forgets about the restrictions on other systems.
A second thing to know about other iconv
implementations is that
the number of available conversions is often very limited. Some
implementations provide, in the standard release (not special
international or developer releases), at most 100 to 200 conversion
possibilities. This does not mean 200 different character sets are
supported; for example, conversions from one character set to a set of 10
others might count as 10 conversions. Together with the other direction
this makes 20 conversion possibilities used up by one character set. One
can imagine the thin coverage these platform provide. Some Unix vendors
even provide only a handful of conversions, which renders them useless for
almost all uses.
This directly leads to a third and probably the most problematic point.
The way the iconv
conversion functions are implemented on all
known Unix systems and the availability of the conversion functions from
character set A to B and the conversion from
B to C does not imply that the
conversion from A to C is available.
This might not seem unreasonable and problematic at first, but it is a
quite big problem as one will notice shortly after hitting it. To show
the problem we assume to write a program that has to convert from
A to C. A call like
cd = iconv_open ("C", "A");
fails according to the assumption above. But what does the program do now? The conversion is necessary; therefore, simply giving up is not an option.
This is a nuisance. The iconv
function should take care of this.
But how should the program proceed from here on? If it tries to convert
to character set B, first the two iconv_open
calls
cd1 = iconv_open ("B", "A");
and
cd2 = iconv_open ("C", "B");
will succeed, but how to find B?
Unfortunately, the answer is: there is no general solution. On some systems guessing might help. On those systems most character sets can convert to and from UTF-8 encoded ISO 10646 or Unicode text. Beside this only some very system-specific methods can help. Since the conversion functions come from loadable modules and these modules must be stored somewhere in the filesystem, one could try to find them and determine from the available file which conversions are available and whether there is an indirect route from A to C.
This example shows one of the design errors of iconv
mentioned
above. It should at least be possible to determine the list of available
conversion programmatically so that if iconv_open
says there is no
such conversion, one could make sure this also is true for indirect
routes.
iconv
Implementation in the GNU C libraryAfter reading about the problems of iconv
implementations in the
last section it is certainly good to note that the implementation in
the GNU C library has none of the problems mentioned above. What
follows is a step-by-step analysis of the points raised above. The
evaluation is based on the current state of the development (as of
January 1999). The development of the iconv
functions is not
complete, but basic functionality has solidified.
The GNU C library's iconv
implementation uses shared loadable
modules to implement the conversions. A very small number of
conversions are built into the library itself but these are only rather
trivial conversions.
All the benefits of loadable modules are available in the GNU C library
implementation. This is especially appealing since the interface is
well documented (see below), and it, therefore, is easy to write new
conversion modules. The drawback of using loadable objects is not a
problem in the GNU C library, at least on ELF systems. Since the
library is able to load shared objects even in statically linked
binaries, static linking need not be forbidden in case one wants to use
iconv
.
The second mentioned problem is the number of supported conversions. Currently, the GNU C library supports more than 150 character sets. The way the implementation is designed the number of supported conversions is greater than 22350 (150 times 149). If any conversion from or to a character set is missing, it can be added easily.
Particularly impressive as it may be, this high number is due to the
fact that the GNU C library implementation of iconv
does not have
the third problem mentioned above (i.e., whenever there is a conversion
from a character set A to B and from
B to C it is always possible to convert from
A to C directly). If the iconv_open
returns an error and sets errno
to EINVAL
, there is no
known way, directly or indirectly, to perform the wanted conversion.
Triangulation is achieved by providing for each character set a conversion from and to UCS-4 encoded ISO 10646. Using ISO 10646 as an intermediate representation it is possible to triangulate (i.e., convert with an intermediate representation).
There is no inherent requirement to provide a conversion to ISO 10646 for a new character set, and it is also possible to provide other conversions where neither source nor destination character set is ISO 10646. The existing set of conversions is simply meant to cover all conversions that might be of interest.
All currently available conversions use the triangulation method above, making conversion run unnecessarily slow. If, for example, somebody often needs the conversion from ISO-2022-JP to EUC-JP, a quicker solution would involve direct conversion between the two character sets, skipping the input to ISO 10646 first. The two character sets of interest are much more similar to each other than to ISO 10646.
In such a situation one easily can write a new conversion and provide it
as a better alternative. The GNU C library iconv
implementation
would automatically use the module implementing the conversion if it is
specified to be more efficient.
gconv-modules
filesAll information about the available conversions comes from a file named
gconv-modules
, which can be found in any of the directories along
the GCONV_PATH
. The gconv-modules
files are line-oriented
text files, where each of the lines has one of the following formats:
alias
define an alias name for a character
set. Two more words are expected on the line. The first word
defines the alias name, and the second defines the original name of the
character set. The effect is that it is possible to use the alias name
in the fromset or toset parameters of iconv_open
and
achieve the same result as when using the real character set name.
This is quite important as a character set has often many different
names. There is normally an official name but this need not correspond to
the most popular name. Beside this many character sets have special
names that are somehow constructed. For example, all character sets
specified by the ISO have an alias of the form ISO-IR-nnn
where nnn is the registration number. This allows programs that
know about the registration number to construct character set names and
use them in iconv_open
calls. More on the available names and
aliases follows below.
module
introduce an available conversion
module. These lines must contain three or four more words.
The first word specifies the source character set, the second word the
destination character set of conversion implemented in this module, and
the third word is the name of the loadable module. The filename is
constructed by appending the usual shared object suffix (normally
.so
) and this file is then supposed to be found in the same
directory the gconv-modules
file is in. The last word on the line,
which is optional, is a numeric value representing the cost of the
conversion. If this word is missing, a cost of 1 is assumed. The
numeric value itself does not matter that much; what counts are the
relative values of the sums of costs for all possible conversion paths.
Below is a more precise description of the use of the cost value.
Returning to the example above where one has written a module to directly
convert from ISO-2022-JP to EUC-JP and back. All that has to be done is
to put the new module, let its name be ISO2022JP-EUCJP.so, in a directory
and add a file gconv-modules
with the following content in the
same directory:
module ISO-2022-JP// EUC-JP// ISO2022JP-EUCJP 1 module EUC-JP// ISO-2022-JP// ISO2022JP-EUCJP 1
To see why this is sufficient, it is necessary to understand how the
conversion used by iconv
(and described in the descriptor) is
selected. The approach to this problem is quite simple.
At the first call of the iconv_open
function the program reads
all available gconv-modules
files and builds up two tables: one
containing all the known aliases and another that contains the
information about the conversions and which shared object implements
them.
iconv
The set of available conversions form a directed graph with weighted
edges. The weights on the edges are the costs specified in the
gconv-modules
files. The iconv_open
function uses an
algorithm suitable for search for the best path in such a graph and so
constructs a list of conversions that must be performed in succession
to get the transformation from the source to the destination character
set.
Explaining why the above gconv-modules
files allows the
iconv
implementation to resolve the specific ISO-2022-JP to
EUC-JP conversion module instead of the conversion coming with the
library itself is straightforward. Since the latter conversion takes two
steps (from ISO-2022-JP to ISO 10646 and then from ISO 10646 to
EUC-JP), the cost is 1+1 = 2. The above gconv-modules
file, however, specifies that the new conversion modules can perform this
conversion with only the cost of 1.
A mysterious item about the gconv-modules
file above (and also
the file coming with the GNU C library) are the names of the character
sets specified in the module
lines. Why do almost all the names
end in //
? And this is not all: the names can actually be
regular expressions. At this point in time this mystery should not be
revealed, unless you have the relevant spell-casting materials: ashes
from an original DOS 6.2 boot disk burnt in effigy, a crucifix
blessed by St. Emacs, assorted herbal roots from Central America, sand
from Cebu, etc. Sorry! The part of the implementation where
this is used is not yet finished. For now please simply follow the
existing examples. It'll become clearer once it is. -drepper
A last remark about the gconv-modules
is about the names not
ending with //
. A character set named INTERNAL
is often
mentioned. From the discussion above and the chosen name it should have
become clear that this is the name for the representation used in the
intermediate step of the triangulation. We have said that this is UCS-4
but actually that is not quite right. The UCS-4 specification also
includes the specification of the byte ordering used. Since a UCS-4 value
consists of four bytes, a stored value is effected by byte ordering. The
internal representation is not the same as UCS-4 in case the byte
ordering of the processor (or at least the running process) is not the
same as the one required for UCS-4. This is done for performance reasons
as one does not want to perform unnecessary byte-swapping operations if
one is not interested in actually seeing the result in UCS-4. To avoid
trouble with endianess, the internal representation consistently is named
INTERNAL
even on big-endian systems where the representations are
identical.
iconv
module data structuresSo far this section has described how modules are located and considered to be used. What remains to be described is the interface of the modules so that one can write new ones. This section describes the interface as it is in use in January 1999. The interface will change a bit in the future but, with luck, only in an upwardly compatible way.
The definitions necessary to write new modules are publicly available
in the non-standard header gconv.h
. The following text,
therefore, describes the definitions from this header file. First,
however, it is necessary to get an overview.
From the perspective of the user of iconv
the interface is quite
simple: the iconv_open
function returns a handle that can be used
in calls to iconv
, and finally the handle is freed with a call to
iconv_close
. The problem is that the handle has to be able to
represent the possibly long sequences of conversion steps and also the
state of each conversion since the handle is all that is passed to the
iconv
function. Therefore, the data structures are really the
elements necessary to understanding the implementation.
We need two different kinds of data structures. The first describes the
conversion and the second describes the state etc. There are really two
type definitions like this in gconv.h
.
struct __gconv_step | Data type |
This data structure describes one conversion a module can perform. For
each function in a loaded module with conversion functions there is
exactly one object of this type. This object is shared by all users of
the conversion (i.e., this object does not contain any information
corresponding to an actual conversion; it only describes the conversion
itself).
|
struct __gconv_step_data | Data type |
This is the data structure that contains the information specific to
each use of the conversion functions.
|
iconv
module interfacesWith the knowledge about the data structures we now can describe the conversion function itself. To understand the interface a bit of knowledge is necessary about the functionality in the C library that loads the objects with the conversions.
It is often the case that one conversion is used more than once (i.e.,
there are several iconv_open
calls for the same set of character
sets during one program run). The mbsrtowcs
et.al. functions in
the GNU C library also use the iconv
functionality, which
increases the number of uses of the same functions even more.
Because of this multiple use of conversions, the modules do not get
loaded exclusively for one conversion. Instead a module once loaded can
be used by an arbitrary number of iconv
or mbsrtowcs
calls
at the same time. The splitting of the information between conversion-
function-specific information and conversion data makes this possible.
The last section showed the two data structures used to do this.
This is of course also reflected in the interface and semantics of the functions that the modules must provide. There are three functions that must have the following names:
gconv_init
gconv_init
function initializes the conversion function
specific data structure. This very same object is shared by all
conversions that use this conversion and, therefore, no state information
about the conversion itself must be stored in here. If a module
implements more than one conversion, the gconv_init
function will
be called multiple times.
gconv_end
gconv_end
function is responsible for freeing all resources
allocated by the gconv_init
function. If there is nothing to do,
this function can be missing. Special care must be taken if the module
implements more than one conversion and the gconv_init
function
does not allocate the same resources for all conversions.
gconv
gconv_init
and the conversion data, specific to
this use of the conversion functions.
There are three data types defined for the three module interface functions and these define the interface.
int (*__gconv_init_fct) (struct __gconv_step *) | Data type |
This specifies the interface of the initialization function of the
module. It is called exactly once for each conversion the module
implements.
As explained in the description of the
If the initialization function needs to communicate some information
to the conversion function, this communication can happen using the
#define MIN_NEEDED_FROM 1 #define MAX_NEEDED_FROM 4 #define MIN_NEEDED_TO 4 #define MAX_NEEDED_TO 4 int gconv_init (struct __gconv_step *step) { /* Determine which direction. */ struct iso2022jp_data *new_data; enum direction dir = illegal_dir; enum variant var = illegal_var; int result; if (__strcasecmp (step->__from_name, "ISO-2022-JP//") == 0) { dir = from_iso2022jp; var = iso2022jp; } else if (__strcasecmp (step->__to_name, "ISO-2022-JP//") == 0) { dir = to_iso2022jp; var = iso2022jp; } else if (__strcasecmp (step->__from_name, "ISO-2022-JP-2//") == 0) { dir = from_iso2022jp; var = iso2022jp2; } else if (__strcasecmp (step->__to_name, "ISO-2022-JP-2//") == 0) { dir = to_iso2022jp; var = iso2022jp2; } result = __GCONV_NOCONV; if (dir != illegal_dir) { new_data = (struct iso2022jp_data *) malloc (sizeof (struct iso2022jp_data)); result = __GCONV_NOMEM; if (new_data != NULL) { new_data->dir = dir; new_data->var = var; step->__data = new_data; if (dir == from_iso2022jp) { step->__min_needed_from = MIN_NEEDED_FROM; step->__max_needed_from = MAX_NEEDED_FROM; step->__min_needed_to = MIN_NEEDED_TO; step->__max_needed_to = MAX_NEEDED_TO; } else { step->__min_needed_from = MIN_NEEDED_TO; step->__max_needed_from = MAX_NEEDED_TO; step->__min_needed_to = MIN_NEEDED_FROM; step->__max_needed_to = MAX_NEEDED_FROM + 2; } /* Yes, this is a stateful encoding. */ step->__stateful = 1; result = __GCONV_OK; } } return result; } The function first checks which conversion is wanted. The module from which this function is taken implements four different conversions; which one is selected can be determined by comparing the names. The comparison should always be done without paying attention to the case. Next, a data structure, which contains the necessary information about
which conversion is selected, is allocated. The data structure
One interesting thing is the initialization of the The possible return values of the initialization function are:
|
The function called before the module is unloaded is significantly easier. It often has nothing at all to do; in which case it can be left out completely.
void (*__gconv_end_fct) (struct gconv_step *) | Data type |
The task of this function is to free all resources allocated in the
initialization function. Therefore only the __data element of
the object pointed to by the argument is of interest. Continuing the
example from the initialization function, the finalization function
looks like this:
void gconv_end (struct __gconv_step *data) { free (data->__data); } |
The most important function is the conversion function itself, which can get quite complicated for complex character sets. But since this is not of interest here, we will only describe a possible skeleton for the conversion function.
int (*__gconv_fct) (struct __gconv_step *, struct __gconv_step_data *, const char **, const char *, size_t *, int) | Data type |
The conversion function can be called for two basic reason: to convert
text or to reset the state. From the description of the iconv
function it can be seen why the flushing mode is necessary. What mode
is selected is determined by the sixth argument, an integer. This
argument being nonzero means that flushing is selected.
Common to both modes is where the output buffer can be found. The
information about this buffer is stored in the conversion step data. A
pointer to this information is passed as the second argument to this
function. The description of the What has to be done for flushing depends on the source character set.
If the source character set is not stateful, nothing has to be done.
Otherwise the function has to emit a byte sequence to bring the state
object into the initial state. Once this all happened the other
conversion modules in the chain of conversions have to get the same
chance. Whether another step follows can be determined from the
The more interesting mode is when actual text has to be converted. The first step in this case is to convert as much text as possible from the input buffer and store the result in the output buffer. The start of the input buffer is determined by the third argument, which is a pointer to a pointer variable referencing the beginning of the buffer. The fourth argument is a pointer to the byte right after the last byte in the buffer. The conversion has to be performed according to the current state if the
character set is stateful. The state is stored in an object pointed to
by the What now happens depends on whether this step is the last one. If it is
the last step, the only thing that has to be done is to update the
In case the step is not the last one, the later conversion functions have
to get a chance to do their work. Therefore, the appropriate conversion
function has to be called. The information about the functions is
stored in the conversion data structures, passed as the first parameter.
This information and the step data are stored in arrays, so the next
element in both cases can be found by simple pointer arithmetic:
int gconv (struct __gconv_step *step, struct __gconv_step_data *data, const char **inbuf, const char *inbufend, size_t *written, int do_flush) { struct __gconv_step *next_step = step + 1; struct __gconv_step_data *next_data = data + 1; ... The next_step->__fct (next_step, next_data, &outerr, outbuf, written, 0) But this is not yet all. Once the function call returns the conversion
function might have some more to do. If the return value of the function
is A requirement for the conversion function is that the input buffer pointer (the third argument) always point to the last character that was put in converted form into the output buffer. This is trivially true after the conversion performed in the current step, but if the conversion functions deeper downstream stop prematurely, not all characters from the output buffer are consumed and, therefore, the input buffer pointers must be backed off to the right position. Correcting the input buffers is easy to do if the input and output character sets have a fixed width for all characters. In this situation we can compute how many characters are left in the output buffer and, therefore, can correct the input buffer pointer appropriately with a similar computation. Things are getting tricky if either character set has characters represented with variable length byte sequences, and it gets even more complicated if the conversion has to take care of the state. In these cases the conversion has to be performed once again, from the known state before the initial conversion (i.e., if necessary the state of the conversion has to be reset and the conversion loop has to be executed again). The difference now is that it is known how much input must be created, and the conversion can stop before converting the first unused character. Once this is done the input buffer pointers must be updated again and the function can return. One final thing should be mentioned. If it is necessary for the
conversion to know whether it is the first invocation (in case a prolog
has to be emitted), the conversion function should increment the
The return value must be one of the following values:
The following example provides a framework for a conversion function.
In case a new conversion has to be written the holes in this
implementation have to be filled and that is it.
int gconv (struct __gconv_step *step, struct __gconv_step_data *data, const char **inbuf, const char *inbufend, size_t *written, int do_flush) { struct __gconv_step *next_step = step + 1; struct __gconv_step_data *next_data = data + 1; gconv_fct fct = next_step->__fct; int status; /* If the function is called with no input this means we have to reset to the initial state. The possibly partly converted input is dropped. */ if (do_flush) { status = __GCONV_OK; /* Possible emit a byte sequence which put the state object into the initial state. */ /* Call the steps down the chain if there are any but only if we successfully emitted the escape sequence. */ if (status == __GCONV_OK && ! data->__is_last) status = fct (next_step, next_data, NULL, NULL, written, 1); } else { /* We preserve the initial values of the pointer variables. */ const char *inptr = *inbuf; char *outbuf = data->__outbuf; char *outend = data->__outbufend; char *outptr; do { /* Remember the start value for this round. */ inptr = *inbuf; /* The outbuf buffer is empty. */ outptr = outbuf; /* For stateful encodings the state must be safe here. */ /* Run the conversion loop. |
This information should be sufficient to write new modules. Anybody doing so should also take a look at the available source code in the GNU C library sources. It contains many examples of working and optimized modules.
Different countries and cultures have varying conventions for how to communicate. These conventions range from very simple ones, such as the format for representing dates and times, to very complex ones, such as the language spoken.
Internationalization of software means programming it to be able to adapt to the user's favorite conventions. In ISO C, internationalization works by means of locales. Each locale specifies a collection of conventions, one convention for each purpose. The user chooses a set of conventions by specifying a locale (via environment variables).
All programs inherit the chosen locale as part of their environment. Provided the programs are written to obey the choice of locale, they will follow the conventions preferred by the user.
Each locale specifies conventions for several purposes, including the following:
Some aspects of adapting to the specified locale are handled
automatically by the library subroutines. For example, all your program
needs to do in order to use the collating sequence of the chosen locale
is to use strcoll
or strxfrm
to compare strings.
Other aspects of locales are beyond the comprehension of the library. For example, the library can't automatically translate your program's output messages into other languages. The only way you can support output in the user's favorite language is to program this more or less by hand. The C library provides functions to handle translations for multiple languages easily.
This chapter discusses the mechanism by which you can modify the current locale. The effects of the current locale on specific library functions are discussed in more detail in the descriptions of those functions.
The simplest way for the user to choose a locale is to set the
environment variable LANG
. This specifies a single locale to use
for all purposes. For example, a user could specify a hypothetical
locale named espana-castellano
to use the standard conventions of
most of Spain.
The set of locales supported depends on the operating system you are
using, and so do their names. We can't make any promises about what
locales will exist, except for one standard locale called C
or
POSIX
. Later we will describe how to construct locales.
A user also has the option of specifying different locales for different purposes--in effect, choosing a mixture of multiple locales.
For example, the user might specify the locale espana-castellano
for most purposes, but specify the locale usa-english
for
currency formatting. This might make sense if the user is a
Spanish-speaking American, working in Spanish, but representing monetary
amounts in US dollars.
Note that both locales espana-castellano
and usa-english
,
like all locales, would include conventions for all of the purposes to
which locales apply. However, the user can choose to use each locale
for a particular subset of those purposes.
The purposes that locales serve are grouped into categories, so
that a user or a program can choose the locale for each category
independently. Here is a table of categories; each name is both an
environment variable that a user can set, and a macro name that you can
use as an argument to setlocale
.
LC_COLLATE
strcoll
and strxfrm
); see Collation Functions.
LC_CTYPE
LC_MONETARY
LC_NUMERIC
LC_TIME
LC_MESSAGES
LC_ALL
setlocale
to set a single locale for all purposes. Setting
this environment variable overwrites all selections by the other
LC_*
variables or LANG
.
LANG
When developing the message translation functions it was felt that the
functionality provided by the variables above is not sufficient. For
example, it should be possible to specify more than one locale name.
Take a Swedish user who better speaks German than English, and a program
whose messages are output in English by default. It should be possible
to specify that the first choice of language is Swedish, the second
German, and if this also fails to use English. This is
possible with the variable LANGUAGE
. For further description of
this GNU extension see Using gettextized software.
A C program inherits its locale environment variables when it starts up.
This happens automatically. However, these variables do not
automatically control the locale used by the library functions, because
ISO C says that all programs start by default in the standard C
locale. To use the locales specified by the environment, you must call
setlocale
. Call it as follows:
setlocale (LC_ALL, "");
to select a locale based on the user choice of the appropriate environment variables.
You can also use setlocale
to specify a particular locale, for
general use or for a specific category.
The symbols in this section are defined in the header file locale.h
.
char * setlocale (int category, const char *locale) | Function |
The function setlocale sets the current locale for category
category to locale. A list of all the locales the system
provides can be created by running
locale -a If category is You can also use this function to find out the current locale by passing
a null pointer as the locale argument. In this case,
The string returned by You should not modify the string returned by When you read the current locale for category To be sure you can use the returned string encoding the currently selected locale at a later time, you must make a copy of the string. It is not guaranteed that the returned pointer remains valid over time. When the locale argument is not a null pointer, the string returned
by If you specify an empty string for locale, this means to read the appropriate environment variable and use its value to select the locale for category. If a nonempty string is given for locale, then the locale of that name is used if possible. If you specify an invalid locale name, |
Here is an example showing how you might use setlocale
to
temporarily switch to a new locale.
#include <stddef.h>
#include <locale.h>
#include <stdlib.h>
#include <string.h>
void
with_other_locale (char *new_locale,
void (*subroutine) (int),
int argument)
{
char *old_locale, *saved_locale;
/* Get the name of the current locale. */
old_locale = setlocale (LC_ALL, NULL);
/* Copy the name so it won't be clobbered by setlocale
. */
saved_locale = strdup (old_locale);
if (saved_locale == NULL)
fatal ("Out of memory");
/* Now change the locale and do some stuff with it. */
setlocale (LC_ALL, new_locale);
(*subroutine) (argument);
/* Restore the original locale. */
setlocale (LC_ALL, saved_locale);
free (saved_locale);
}
Portability Note: Some ISO C systems may define additional
locale categories, and future versions of the library will do so. For
portability, assume that any symbol beginning with LC_
might be
defined in locale.h
.
The only locale names you can count on finding on all operating systems are these three standard ones:
"C"
"POSIX"
""
Defining and installing named locales is normally a responsibility of the system administrator at your site (or the person who installed the GNU C library). It is also possible for the user to create private locales. All this will be discussed later when describing the tool to do so.
If your program needs to use something other than the C
locale,
it will be more portable if you use whatever locale the user specifies
with the environment, rather than trying to specify some non-standard
locale explicitly by name. Remember, different machines might have
different sets of locales installed.
There are several ways to access locale information. The simplest way is to let the C library itself do the work. Several of the functions in this library implicitly access the locale data, and use what information is provided by the currently selected locale. This is how the locale model is meant to work normally.
As an example take the strftime
function, which is meant to nicely
format date and time information (see Formatting Calendar Time).
Part of the standard information contained in the LC_TIME
category is the names of the months. Instead of requiring the
programmer to take care of providing the translations the
strftime
function does this all by itself. %A
in the format string is replaced by the appropriate weekday
name of the locale currently selected by LC_TIME
. This is an
easy example, and wherever possible functions do things automatically
in this way.
But there are quite often situations when there is simply no function
to perform the task, or it is simply not possible to do the work
automatically. For these cases it is necessary to access the
information in the locale directly. To do this the C library provides
two functions: localeconv
and nl_langinfo
. The former is
part of ISO C and therefore portable, but has a brain-damaged
interface. The second is part of the Unix interface and is portable in
as far as the system follows the Unix standards.
localeconv
.
nl_langinfo
.
localeconv
: It is portable but ...Together with the setlocale
function the ISO C people
invented the localeconv
function. It is a masterpiece of poor
design. It is expensive to use, not extendable, and not generally
usable as it provides access to only LC_MONETARY
and
LC_NUMERIC
related information. Nevertheless, if it is
applicable to a given situation it should be used since it is very
portable. The function strfmon
formats monetary amounts
according to the selected locale using this information.
struct lconv * localeconv (void) | Function |
The localeconv function returns a pointer to a structure whose
components contain information about how numeric and monetary values
should be formatted in the current locale.
You should not modify the structure or its contents. The structure might
be overwritten by subsequent calls to |
struct lconv | Data Type |
localeconv 's return value is of this data type. Its elements are
described in the following subsections.
|
If a member of the structure struct lconv
has type char
,
and the value is CHAR_MAX
, it means that the current locale has
no value for that parameter.
$
).
These are the standard members of struct lconv
; there may be
others.
char *decimal_point
char *mon_decimal_point
C
locale, the
value of decimal_point
is "."
, and the value of
mon_decimal_point
is ""
.
char *thousands_sep
char *mon_thousands_sep
C
locale, both members have a value of
""
(the empty string).
char *grouping
char *mon_grouping
grouping
applies to non-monetary quantities
and mon_grouping
applies to monetary quantities. Use either
thousands_sep
or mon_thousands_sep
to separate the digit
groups.
Each member of these strings is to be interpreted as an integer value of
type char
. Successive numbers (from left to right) give the
sizes of successive groups (from right to left, starting at the decimal
point.) The last member is either 0
, in which case the previous
member is used over and over again for all the remaining groups, or
CHAR_MAX
, in which case there is no more grouping--or, put
another way, any remaining digits form one large group without
separators.
For example, if grouping
is "\04\03\02"
, the correct
grouping for the number 123456787654321
is 12
, 34
,
56
, 78
, 765
, 4321
. This uses a group of 4
digits at the end, preceded by a group of 3 digits, preceded by groups
of 2 digits (as many as needed). With a separator of ,
, the
number would be printed as 12,34,56,78,765,4321
.
A value of "\03"
indicates repeated groups of three digits, as
normally used in the U.S.
In the standard C
locale, both grouping
and
mon_grouping
have a value of ""
. This value specifies no
grouping at all.
char int_frac_digits
char frac_digits
In the standard C
locale, both of these members have the value
CHAR_MAX
, meaning "unspecified". The ISO standard doesn't say
what to do when you find this value; we recommend printing no
fractional digits. (This locale also specifies the empty string for
mon_decimal_point
, so printing any fractional digits would be
confusing!)
These members of the struct lconv
structure specify how to print
the symbol to identify a monetary value--the international analog of
$
for US dollars.
Each country has two standard currency symbols. The local currency symbol is used commonly within the country, while the international currency symbol is used internationally to refer to that country's currency when it is necessary to indicate the country unambiguously.
For example, many countries use the dollar as their monetary unit, and when dealing with international currencies it's important to specify that one is dealing with (say) Canadian dollars instead of U.S. dollars or Australian dollars. But when the context is known to be Canada, there is no need to make this explicit--dollar amounts are implicitly assumed to be in Canadian dollars.
char *currency_symbol
In the standard C
locale, this member has a value of ""
(the empty string), meaning "unspecified". The ISO standard doesn't
say what to do when you find this value; we recommend you simply print
the empty string as you would print any other string pointed to by this
variable.
char *int_curr_symbol
The value of int_curr_symbol
should normally consist of a
three-letter abbreviation determined by the international standard
ISO 4217 Codes for the Representation of Currency and Funds,
followed by a one-character separator (often a space).
In the standard C
locale, this member has a value of ""
(the empty string), meaning "unspecified". We recommend you simply print
the empty string as you would print any other string pointed to by this
variable.
char p_cs_precedes
char n_cs_precedes
char int_p_cs_precedes
char int_n_cs_precedes
1
if the currency_symbol
or
int_curr_symbol
strings should precede the value of a monetary
amount, or 0
if the strings should follow the value. The
p_cs_precedes
and int_p_cs_precedes
members apply to
positive amounts (or zero), and the n_cs_precedes
and
int_n_cs_precedes
members apply to negative amounts.
In the standard C
locale, all of these members have a value of
CHAR_MAX
, meaning "unspecified". The ISO standard doesn't say
what to do when you find this value. We recommend printing the
currency symbol before the amount, which is right for most countries.
In other words, treat all nonzero values alike in these members.
The members with the int_
prefix apply to the
int_curr_symbol
while the other two apply to
currency_symbol
.
char p_sep_by_space
char n_sep_by_space
char int_p_sep_by_space
char int_n_sep_by_space
1
if a space should appear between the
currency_symbol
or int_curr_symbol
strings and the
amount, or 0
if no space should appear. The
p_sep_by_space
and int_p_sep_by_space
members apply to
positive amounts (or zero), and the n_sep_by_space
and
int_n_sep_by_space
members apply to negative amounts.
In the standard C
locale, all of these members have a value of
CHAR_MAX
, meaning "unspecified". The ISO standard doesn't say
what you should do when you find this value; we suggest you treat it as
1 (print a space). In other words, treat all nonzero values alike in
these members.
The members with the int_
prefix apply to the
int_curr_symbol
while the other two apply to
currency_symbol
. There is one specialty with the
int_curr_symbol
, though. Since all legal values contain a space
at the end the string one either printf this space (if the currency
symbol must appear in front and must be separated) or one has to avoid
printing this character at all (especially when at the end of the
string).
These members of the struct lconv
structure specify how to print
the sign (if any) of a monetary value.
char *positive_sign
char *negative_sign
In the standard C
locale, both of these members have a value of
""
(the empty string), meaning "unspecified".
The ISO standard doesn't say what to do when you find this value; we
recommend printing positive_sign
as you find it, even if it is
empty. For a negative value, print negative_sign
as you find it
unless both it and positive_sign
are empty, in which case print
-
instead. (Failing to indicate the sign at all seems rather
unreasonable.)
char p_sign_posn
char n_sign_posn
char int_p_sign_posn
char int_n_sign_posn
positive_sign
or negative_sign
.) The possible values are
as follows:
0
1
2
3
4
CHAR_MAX
C
locale.
The ISO standard doesn't say what you should do when the value is
CHAR_MAX
. We recommend you print the sign after the currency
symbol.
The members with the int_
prefix apply to the
int_curr_symbol
while the other two apply to
currency_symbol
.
When writing the X/Open Portability Guide the authors realized that the
localeconv
function is not enough to provide reasonable access to
locale information. The information which was meant to be available
in the locale (as later specified in the POSIX.1 standard) requires more
ways to access it. Therefore the nl_langinfo
function
was introduced.
char * nl_langinfo (nl_item item) | Function |
The nl_langinfo function can be used to access individual
elements of the locale categories. Unlike the localeconv
function, which returns all the information, nl_langinfo
lets the caller select what information it requires. This is very
fast and it is not a problem to call this function multiple times.
A second advantage is that in addition to the numeric and monetary
formatting information, information from the
The type
The file Note that the return value for any valid argument can be used for
in all situations (with the possible exception of the am/pm time formatting
codes). If the user has not selected any locale for the
appropriate category, If the argument item is not valid, a pointer to an empty string is returned. |
An example of nl_langinfo
usage is a function which has to
print a given date and time in a locale-specific way. At first one
might think that, since strftime
internally uses the locale
information, writing something like the following is enough:
size_t i18n_time_n_data (char *s, size_t len, const struct tm *tp) { return strftime (s, len, "%X %D", tp); }
The format contains no weekday or month names and therefore is
internationally usable. Wrong! The output produced is something like
"hh:mm:ss MM/DD/YY"
. This format is only recognizable in the
USA. Other countries use different formats. Therefore the function
should be rewritten like this:
size_t i18n_time_n_data (char *s, size_t len, const struct tm *tp) { return strftime (s, len, nl_langinfo (D_T_FMT), tp); }
Now it uses the date and time format of the locale selected when the program runs. If the user selects the locale correctly there should never be a misunderstanding over the time and date format.
We have seen that the structure returned by localeconv
as well as
the values given to nl_langinfo
allow you to retrieve the various
pieces of locale-specific information to format numbers and monetary
amounts. We have also seen that the underlying rules are quite complex.
Therefore the X/Open standards introduce a function which uses such locale information, making it easier for the user to format numbers according to these rules.
ssize_t strfmon (char *s, size_t maxsize, const char *format, ...) | Function |
The strfmon function is similar to the strftime function
in that it takes a buffer, its size, a format string,
and values to write into the buffer as text in a form specified
by the format string. Like strftime , the function
also returns the number of bytes written into the buffer.
There are two differences:
The next part of a specification is an optional field width. If no
width is specified 0 is taken. During output, the function first
determines how much space is required. If it requires at least as many
characters as given by the field width, it is output using as much space
as necessary. Otherwise, it is extended to use the full width by
filling with the space character. The presence or absence of the
So far the format looks familiar, being similar to the The second optional field starts with a As a GNU extension, the Finally, the last component is a format specifier. There are three specifiers defined:
As for The return value of the function is the number of characters stored in
s, including the terminating |
A few examples should make clear how the function works. It is
assumed that all the following pieces of code are executed in a program
which uses the USA locale (en_US
). The simplest
form of the format is this:
strfmon (buf, 100, "@%n@%n@%n@", 123.45, -567.89, 12345.678);
The output produced is
"@$123.45@-$567.89@$12,345.68@"
We can notice several things here. First, the widths of the output
numbers are different. We have not specified a width in the format
string, and so this is no wonder. Second, the third number is printed
using thousands separators. The thousands separator for the
en_US
locale is a comma. The number is also rounded.
.678 is rounded to .68 since the format does not specify a
precision and the default value in the locale is 2. Finally,
note that the national currency symbol is printed since %n
was
used, not i
. The next example shows how we can align the output.
strfmon (buf, 100, "@%=*11n@%=*11n@%=*11n@", 123.45, -567.89, 12345.678);
The output this time is:
"@ $123.45@ -$567.89@ $12,345.68@"
Two things stand out. Firstly, all fields have the same width (eleven
characters) since this is the width given in the format and since no
number required more characters to be printed. The second important
point is that the fill character is not used. This is correct since the
white space was not used to achieve a precision given by a #
modifier, but instead to fill to the given width. The difference
becomes obvious if we now add a width specification.
strfmon (buf, 100, "@%=*11#5n@%=*11#5n@%=*11#5n@", 123.45, -567.89, 12345.678);
The output is
"@ $***123.45@-$***567.89@ $12,456.68@"
Here we can see that all the currency symbols are now aligned, and that
the space between the currency sign and the number is filled with the
selected fill character. Note that although the width is selected to be
5 and 123.45 has three digits left of the decimal point,
the space is filled with three asterisks. This is correct since, as
explained above, the width does not include the positions used to store
thousands separators. One last example should explain the remaining
functionality.
strfmon (buf, 100, "@%=0(16#5.3i@%=0(16#5.3i@%=0(16#5.3i@", 123.45, -567.89, 12345.678);
This rather complex format string produces the following output:
"@ USD 000123,450 @(USD 000567.890)@ USD 12,345.678 @"
The most noticeable change is the alternative way of representing
negative numbers. In financial circles this is often done using
parentheses, and this is what the (
flag selected. The fill
character is now 0
. Note that this 0
character is not
regarded as a numeric zero, and therefore the first and second numbers
are not printed using a thousands separator. Since we used the format
specifier i
instead of n
, the international form of the
currency symbol is used. This is a four letter string, in this case
"USD "
. The last point is that since the precision right of the
decimal point is selected to be three, the first and second numbers are
printed with an extra zero at the end and the third number is printed
without rounding.
Some non GUI programs ask a yes-or-no question. If the messages (especially the questions) are translated into foreign languages, be sure that you localize the answers too. It would be very bad habit to ask a question in one language and request the answer in another, often English.
The GNU C library contains rpmatch
to give applications easy
access to the corresponding locale definitions.
int rpmatch (const char *response) | Function |
The function rpmatch checks the string in response whether
or not it is a correct yes-or-no answer and if yes, which one. The
check uses the YESEXPR and NOEXPR data in the
LC_MESSAGES category of the currently selected locale. The
return value is as follows:
This function is not standardized but available beside in GNU libc at least also in the IBM AIX library. |
This function would normally be used like this:
... /* Use a safe default. */ _Bool doit = false; fputs (gettext ("Do you really want to do this? "), stdout); fflush (stdout); /* Prepare thegetline
call. */ line = NULL; len = 0; while (getline (&line, &len, stdout) >= 0) { /* Check the response. */ int res = rpmatch (line); if (res >= 0) { /* We got a definitive answer. */ if (res > 0) doit = true; break; } } /* Free whatgetline
allocated. */ free (line);
Note that the loop continues until an read error is detected or until a definitive (positive or negative) answer is read.
The program's interface with the human should be designed in a way to ease the human the task. One of the possibilities is to use messages in whatever language the user prefers.
Printing messages in different languages can be implemented in different ways. One could add all the different languages in the source code and add among the variants every time a message has to be printed. This is certainly no good solution since extending the set of languages is difficult (the code must be changed) and the code itself can become really big with dozens of message sets.
A better solution is to keep the message sets for each language are kept in separate files which are loaded at runtime depending on the language selection of the user.
The GNU C Library provides two different sets of functions to support
message translation. The problem is that neither of the interfaces is
officially defined by the POSIX standard. The catgets
family of
functions is defined in the X/Open standard but this is derived from
industry decisions and therefore not necessarily based on reasonable
decisions.
As mentioned above the message catalog handling provides easy extendibility by using external data files which contain the message translations. I.e., these files contain for each of the messages used in the program a translation for the appropriate language. So the tasks of the message handling functions are
The two approaches mainly differ in the implementation of this last step. The design decisions made for this influences the whole rest.
catgets
family of functions.
gettext
family of functions.
The catgets
functions are based on the simple scheme:
Associate every message to translate in the source code with a unique identifier. To retrieve a message from a catalog file solely the identifier is used.
This means for the author of the program that s/he will have to make sure the meaning of the identifier in the program code and in the message catalogs are always the same.
Before a message can be translated the catalog file must be located. The user of the program must be able to guide the responsible function to find whatever catalog the user wants. This is separated from what the programmer had in mind.
All the types, constants and functions for the catgets
functions
are defined/declared in the nl_types.h
header file.
catgets
function family.
catgets
interface.
catgets
function family
nl_catd catopen (const char *cat_name, int flag) | Function |
The catgets function tries to locate the message data file names
cat_name and loads it when found. The return value is of an
opaque type and can be used in calls to the other functions to refer to
this loaded catalog.
The return value is Locating the catalog file must happen in a way which lets the user of the program influence the decision. It is up to the user to decide about the language to use and sometimes it is useful to use alternate catalog files. All this can be specified by the user by setting some environment variables. The first problem is to find out where all the message catalogs are stored. Every program could have its own place to keep all the different files but usually the catalog files are grouped by languages and the catalogs for all programs are kept in the same place. To tell the /usr/share/locale/%L/%N:/usr/share/locale/%L/LC_MESSAGES/%N First one can see that more than one directory can be specified (with
the usual syntax of separating them by colons). The next things to
observe are the format string,
Using prefix/share/locale/%L/%N:prefix/share/locale/%L/LC_MESSAGES/%N where prefix is given to The remaining problem is to decide which must be used. The value
decides about the substitution of the format elements mentioned above.
First of all the user can specify a path in the message catalog name
(i.e., the name contains a slash character). In this situation the
Otherwise the values of environment variables from the standard
environment are examined (see Standard Environment). Which
variables are examined is decided by the flag parameter of
If flag is zero the The environment variable and the locale name should have a value of the
form The return value of the function is in any case a valid string. Either
it is a translation from a message catalog or it is the same as the
string parameter. So a piece of code to decide whether a
translation actually happened must look like this:
{ char *trans = catgets (desc, set, msg, input_string); if (trans == input_string) { /* Something went wrong. */ } } When an error occurred the global variable errno is set to
While it sometimes can be useful to test for errors programs normally will avoid any test. If the translation is not available it is no big problem if the original, untranslated message is printed. Either the user understands this as well or s/he will look for the reason why the messages are not translated. |
Please note that the currently selected locale does not depend on a call
to the setlocale
function. It is not necessary that the locale
data files for this locale exist and calling setlocale
succeeds.
The catopen
function directly reads the values of the environment
variables.
char * catgets (nl_catd catalog_desc, int set, int message, const char *string) | Function |
The function catgets has to be used to access the massage catalog
previously opened using the catopen function. The
catalog_desc parameter must be a value previously returned by
catopen .
The next two parameters, set and message, reflect the internal organization of the message catalog files. This will be explained in detail below. For now it is interesting to know that a catalog can consists of several set and the messages in each thread are individually numbered using numbers. Neither the set number nor the message number must be consecutive. They can be arbitrarily chosen. But each message (unless equal to another one) must have its own unique pair of set and message number. Since it is not guaranteed that the message catalog for the language selected by the user exists the last parameter string helps to handle this case gracefully. If no matching string can be found string is returned. This means for the programmer that
|
It is somewhat uncomfortable to write a program using the catgets
functions if no supporting functionality is available. Since each
set/message number tuple must be unique the programmer must keep lists
of the messages at the same time the code is written. And the work
between several people working on the same project must be coordinated.
We will see some how these problems can be relaxed a bit (see Common Usage).
int catclose (nl_catd catalog_desc) | Function |
The catclose function can be used to free the resources
associated with a message catalog which previously was opened by a call
to catopen . If the resources can be successfully freed the
function returns 0 . Otherwise it return -1 and the
global variable errno is set. Errors can occur if the catalog
descriptor catalog_desc is not valid in which case errno is
set to EBADF .
|
The only reasonable way the translate all the messages of a function and
store the result in a message catalog file which can be read by the
catopen
function is to write all the message text to the
translator and let her/him translate them all. I.e., we must have a
file with entries which associate the set/message tuple with a specific
translation. This file format is specified in the X/Open standard and
is as follows:
$
followed by a whitespace character are comment and are also ignored.
$set
followed by a whitespace character an additional argument
is required to follow. This argument can either be:
How to use the symbolic names is explained in section Common Usage.
It is an error if a symbol name appears more than once. All following messages are placed in a set with this number.
$delset
followed by a whitespace character an additional argument
is required to follow. This argument can either be:
In both cases all messages in the specified set will be removed. They
will not appear in the output. But if this set is later again selected
with a $set
command again messages could be added and these
messages will appear in the output.
$quote
, the quoting character used for this input file is
changed to the first non-whitespace character following the
$quote
. If no non-whitespace character is present before the
line ends quoting is disable.
By default no quoting character is used. In this mode strings are
terminated with the first unescaped line break. If there is a
$quote
sequence present newline need not be escaped. Instead a
string is terminated with the first unescaped appearance of the quote
character.
A common usage of this feature would be to set the quote character to
"
. Then any appearance of the "
in the strings must
be escaped using the backslash (i.e., \"
must be written).
If the start of the line is a number the message number is obvious. It is an error if the same message number already appeared for this set.
If the leading token was an identifier the message number gets
automatically assigned. The value is the current maximum messages
number for this set plus one. It is an error if the identifier was
already used for a message in this set. It is OK to reuse the
identifier for a message in another thread. How to use the symbolic
identifiers will be explained below (see Common Usage). There is
one limitation with the identifier: it must not be Set
. The
reason will be explained below.
The text of the messages can contain escape characters. The usual bunch
of characters known from the ISO C language are recognized
(\n
, \t
, \v
, \b
, \r
, \f
,
\\
, and \nnn
, where nnn is the octal coding of
a character code).
Important: The handling of identifiers instead of numbers for
the set and messages is a GNU extension. Systems strictly following the
X/Open specification do not have this feature. An example for a message
catalog file is this:
$ This is a leading comment. $quote " $set SetOne 1 Message with ID 1. two " Message with ID \"two\", which gets the value 2 assigned" $set SetTwo $ Since the last set got the number 1 assigned this set has number 2. 4000 "The numbers can be arbitrary, they need not start at one."
This small example shows various aspects:
$
followed by
a whitespace.
"
. Otherwise the quotes in the
message definition would have to be left away and in this case the
message with the identifier two
would loose its leading whitespace.
While this file format is pretty easy it is not the best possible for
use in a running program. The catopen
function would have to
parser the file and handle syntactic errors gracefully. This is not so
easy and the whole process is pretty slow. Therefore the catgets
functions expect the data in another more compact and ready-to-use file
format. There is a special program gencat
which is explained in
detail in the next section.
Files in this other format are not human readable. To be easy to use by programs it is a binary file. But the format is byte order independent so translation files can be shared by systems of arbitrary architecture (as long as they use the GNU C Library).
Details about the binary file format are not important to know since
these files are always created by the gencat
program. The
sources of the GNU C Library also provide the sources for the
gencat
program and so the interested reader can look through
these source files to learn about the file format.
The gencat
program is specified in the X/Open standard and the
GNU implementation follows this specification and so processes
all correctly formed input files. Additionally some extension are
implemented which help to work in a more reasonable way with the
catgets
functions.
The gencat
program can be invoked in two ways:
`gencat [Option]... [Output-File [Input-File]...]`
This is the interface defined in the X/Open standard. If no
Input-File parameter is given input will be read from standard
input. Multiple input files will be read as if they are concatenated.
If Output-File is also missing, the output will be written to
standard output. To provide the interface one is used to from other
programs a second interface is provided.
`gencat [Option]... -o Output-File [Input-File]...`
The option -o
is used to specify the output file and all file
arguments are used as input files.
Beside this one can use -
or /dev/stdin
for
Input-File to denote the standard input. Corresponding one can
use -
and /dev/stdout
for Output-File to denote
standard output. Using -
as a file name is allowed in X/Open
while using the device names is a GNU extension.
The gencat
program works by concatenating all input files and
then merge the resulting collection of message sets with a
possibly existing output file. This is done by removing all messages
with set/message number tuples matching any of the generated messages
from the output file and then adding all the new messages. To
regenerate a catalog file while ignoring the old contents therefore
requires to remove the output file if it exists. If the output is
written to standard output no merging takes place.
The following table shows the options understood by the gencat
program. The X/Open standard does not specify any option for the
program so all of these are GNU extensions.
-V
--version
-h
--help
--new
-H
--header=name
#define
s to associate a name with a
number.
Please note that the generated file only contains the symbols from the input files. If the output is merged with the previous content of the output file the possibly existing symbols from the file(s) which generated the old output files are not in the generated header file.
catgets
interfaceThe catgets
functions can be used in two different ways. By
following slavishly the X/Open specs and not relying on the extension
and by using the GNU extensions. We will take a look at the former
method first to understand the benefits of extensions.
Since the X/Open format of the message catalog files does not allow
symbol names we have to work with numbers all the time. When we start
writing a program we have to replace all appearances of translatable
strings with something like
catgets (catdesc, set, msg, "string")
catgets is retrieved from a call to catopen
which is
normally done once at the program start. The "string"
is the
string we want to translate. The problems start with the set and
message numbers.
In a bigger program several programmers usually work at the same time on the program and so coordinating the number allocation is crucial. Though no two different strings must be indexed by the same tuple of numbers it is highly desirable to reuse the numbers for equal strings with equal translations (please note that there might be strings which are equal in one language but have different translations due to difference contexts).
The allocation process can be relaxed a bit by different set numbers for
different parts of the program. So the number of developers who have to
coordinate the allocation can be reduced. But still lists must be keep
track of the allocation and errors can easily happen. These errors
cannot be discovered by the compiler or the catgets
functions.
Only the user of the program might see wrong messages printed. In the
worst cases the messages are so irritating that they cannot be
recognized as wrong. Think about the translations for "true"
and
"false"
being exchanged. This could result in a disaster.
The problems mentioned in the last section derive from the fact that:
By constantly using symbolic names and by providing a method which maps the string content to a symbolic name (however this will happen) one can prevent both problems above. The cost of this is that the programmer has to write a complete message catalog file while s/he is writing the program itself.
This is necessary since the symbolic names must be mapped to numbers
before the program sources can be compiled. In the last section it was
described how to generate a header containing the mapping of the names.
E.g., for the example message file given in the last section we could
call the gencat
program as follow (assume ex.msg
contains
the sources).
gencat -H ex.h -o ex.cat ex.msg
This generates a header file with the following content:
#define SetTwoSet 0x2 /* ex.msg:8 */ #define SetOneSet 0x1 /* ex.msg:4 */ #define SetOnetwo 0x2 /* ex.msg:6 */
As can be seen the various symbols given in the source file are mangled
to generate unique identifiers and these identifiers get numbers
assigned. Reading the source file and knowing about the rules will
allow to predict the content of the header file (it is deterministic)
but this is not necessary. The gencat
program can take care for
everything. All the programmer has to do is to put the generated header
file in the dependency list of the source files of her/his project and
to add a rules to regenerate the header of any of the input files
change.
One word about the symbol mangling. Every symbol consists of two parts:
the name of the message set plus the name of the message or the special
string Set
. So SetOnetwo
means this macro can be used to
access the translation with identifier two
in the message set
SetOne
.
The other names denote the names of the message sets. The special
string Set
is used in the place of the message identifier.
If in the code the second string of the set SetOne
is used the C
code should look like this:
catgets (catdesc, SetOneSet, SetOnetwo, " Message with ID \"two\", which gets the value 2 assigned")
Writing the function this way will allow to change the message number and even the set number without requiring any change in the C source code. (The text of the string is normally not the same; this is only for this example.)
To illustrate the usual way to work with the symbolic version numbers
here is a little example. Assume we want to write the very complex and
famous greeting program. We start by writing the code as usual:
#include <stdio.h> int main (void) { printf ("Hello, world!\n"); return 0; }
Now we want to internationalize the message and therefore replace the
message with whatever the user wants.
#include <nl_types.h> #include <stdio.h> #include "msgnrs.h" int main (void) { nl_catd catdesc = catopen ("hello.cat", NL_CAT_LOCALE); printf (catgets (catdesc, SetMainSet, SetMainHello, "Hello, world!\n")); catclose (catdesc); return 0; }
We see how the catalog object is opened and the returned descriptor used in the other function calls. It is not really necessary to check for failure of any of the functions since even in these situations the functions will behave reasonable. They simply will be return a translation.
What remains unspecified here are the constants SetMainSet
and
SetMainHello
. These are the symbolic names describing the
message. To get the actual definitions which match the information in
the catalog file we have to create the message catalog source file and
process it using the gencat
program.
$ Messages for the famous greeting program. $quote " $set Main Hello "Hallo, Welt!\n"
Now we can start building the program (assume the message catalog source
file is named hello.msg
and the program source file hello.c
):
% gencat -H msgnrs.h -o hello.cat hello.msg % cat msgnrs.h #define MainSet 0x1 /* hello.msg:4 */ #define MainHello 0x1 /* hello.msg:5 */ % gcc -o hello hello.c -I. % cp hello.cat /usr/share/locale/de/LC_MESSAGES % echo $LC_ALL de % ./hello Hallo, Welt! %
The call of the gencat
program creates the missing header file
msgnrs.h
as well as the message catalog binary. The former is
used in the compilation of hello.c
while the later is placed in a
directory in which the catopen
function will try to locate it.
Please check the LC_ALL
environment variable and the default path
for catopen
presented in the description above.
Sun Microsystems tried to standardize a different approach to message
translation in the Uniforum group. There never was a real standard
defined but still the interface was used in Sun's operation systems.
Since this approach fits better in the development process of free
software it is also used throughout the GNU project and the GNU
gettext
package provides support for this outside the GNU C
Library.
The code of the libintl
from GNU gettext
is the same as
the code in the GNU C Library. So the documentation in the GNU
gettext
manual is also valid for the functionality here. The
following text will describe the library functions in detail. But the
numerous helper programs are not described in this manual. Instead
people should read the GNU gettext
manual
(see GNU gettext utilities).
We will only give a short overview.
Though the catgets
functions are available by default on more
systems the gettext
interface is at least as portable as the
former. The GNU gettext
package can be used wherever the
functions are not available.
gettext
family of functions.
gettext
.
gettext
family of functionsThe paradigms underlying the gettext
approach to message
translations is different from that of the catgets
functions the
basic functionally is equivalent. There are functions of the following
categories:
gettext
uses.
gettext
in GUI programs.
gettext
works.
The gettext
functions have a very simple interface. The most
basic function just takes the string which shall be translated as the
argument and it returns the translation. This is fundamentally
different from the catgets
approach where an extra key is
necessary and the original string is only used for the error case.
If the string which has to be translated is the only argument this of
course means the string itself is the key. I.e., the translation will
be selected based on the original string. The message catalogs must
therefore contain the original strings plus one translation for any such
string. The task of the gettext
function is it to compare the
argument string with the available strings in the catalog and return the
appropriate translation. Of course this process is optimized so that
this process is not more expensive than an access using an atomic key
like in catgets
.
The gettext
approach has some advantages but also some
disadvantages. Please see the GNU gettext
manual for a detailed
discussion of the pros and cons.
All the definitions and declarations for gettext
can be found in
the libintl.h
header file. On systems where these functions are
not part of the C library they can be found in a separate library named
libintl.a
(or accordingly different for shared libraries).
char * gettext (const char *msgid) | Function |
The gettext function searches the currently selected message
catalogs for a string which is equal to msgid. If there is such a
string available it is returned. Otherwise the argument string
msgid is returned.
Please note that all though the return value is Please note that above we wrote "message catalogs" (plural). This is a specialty of the GNU implementation of these functions and we will say more about this when we talk about the ways message catalogs are selected (see Locating gettext catalog). The printf (gettext ("Operation failed: %m\n")); Here the errno value is used in the So there is no easy way to detect a missing message catalog beside comparing the argument string with the result. But it is normally the task of the user to react on missing catalogs. The program cannot guess when a message catalog is really necessary since for a user who speaks the language the program was developed in does not need any translation. |
The remaining two functions to access the message catalog add some
functionality to select a message catalog which is not the default one.
This is important if parts of the program are developed independently.
Every part can have its own message catalog and all of them can be used
at the same time. The C library itself is an example: internally it
uses the gettext
functions but since it must not depend on a
currently selected default message catalog it must specify all ambiguous
information.
char * dgettext (const char *domainname, const char *msgid) | Function |
The dgettext functions acts just like the gettext
function. It only takes an additional first argument domainname
which guides the selection of the message catalogs which are searched
for the translation. If the domainname parameter is the null
pointer the dgettext function is exactly equivalent to
gettext since the default value for the domain name is used.
As for |
char * dcgettext (const char *domainname, const char *msgid, int category) | Function |
The dcgettext adds another argument to those which
dgettext takes. This argument category specifies the last
piece of information needed to localize the message catalog. I.e., the
domain name and the locale category exactly specify which message
catalog has to be used (relative to a given directory, see below).
The dcgettext (domain, string, LC_MESSAGES) instead of
dgettext (domain, string) This also shows which values are expected for the third parameter. One
has to use the available selectors for the categories available in
The As for |
When using the three functions above in a program it is a frequent case
that the msgid argument is a constant string. So it is worth to
optimize this case. Thinking shortly about this one will realize that
as long as no new message catalog is loaded the translation of a message
will not change. This optimization is actually implemented by the
gettext
, dgettext
and dcgettext
functions.
The functions to retrieve the translations for a given message have a remarkable simple interface. But to provide the user of the program still the opportunity to select exactly the translation s/he wants and also to provide the programmer the possibility to influence the way to locate the search for catalogs files there is a quite complicated underlying mechanism which controls all this. The code is complicated the use is easy.
Basically we have two different tasks to perform which can also be
performed by the catgets
functions:
There can be arbitrary many packages installed and they can follow different guidelines for the placement of their files.
This is the functionality required by the specifications for
gettext
and this is also what the catgets
functions are
able to do. But there are some problems unresolved:
de
, german
, or
deutsch
and the program should always react the same.
de_DE.ISO-8859-1
which means German, spoken in Germany,
coded using the ISO 8859-1 character set there is the possibility
that a message catalog matching this exactly is not available. But
there could be a catalog matching de
and if the character set
used on the machine is always ISO 8859-1 there is no reason why this
later message catalog should not be used. (We call this message
inheritance.)
We can divide the configuration actions in two parts: the one is performed by the programmer, the other by the user. We will start with the functions the programmer can use since the user configuration will be based on this.
As the functions described in the last sections already mention separate
sets of messages can be selected by a domain name. This is a
simple string which should be unique for each program part with uses a
separate domain. It is possible to use in one program arbitrary many
domains at the same time. E.g., the GNU C Library itself uses a domain
named libc
while the program using the C Library could use a
domain named foo
. The important point is that at any time
exactly one domain is active. This is controlled with the following
function.
char * textdomain (const char *domainname) | Function |
The textdomain function sets the default domain, which is used in
all future gettext calls, to domainname. Please note that
dgettext and dcgettext calls are not influenced if the
domainname parameter of these functions is not the null pointer.
Before the first call to The function returns the value which is from now on taken as the default
domain. If the system went out of memory the returned value is
If the domainname parameter is the null pointer no new default domain is set. Instead the currently selected default domain is returned. If the domainname parameter is the empty string the default domain
is reset to its initial value, the domain with the name |
char * bindtextdomain (const char *domainname, const char *dirname) | Function |
The bindtextdomain function can be used to specify the directory
which contains the message catalogs for domain domainname for the
different languages. To be correct, this is the directory where the
hierarchy of directories is expected. Details are explained below.
For the programmer it is important to note that the translations which
come with the program have be placed in a directory hierarchy starting
at, say, The If the program which wish to use If the dirname parameter is the null pointer The |
The functions of the gettext
family described so far (and all the
catgets
functions as well) have one problem in the real world
which have been neglected completely in all existing approaches. What
is meant here is the handling of plural forms.
Looking through Unix source code before the time anybody thought about
internationalization (and, sadly, even afterwards) one can often find
code similar to the following:
printf ("%d file%s deleted", n, n == 1 ? "" : "s");
After the first complaints from people internationalizing the code people
either completely avoided formulations like this or used strings like
"file(s)"
. Both look unnatural and should be avoided. First
tries to solve the problem correctly looked like this:
if (n == 1) printf ("%d file deleted", n); else printf ("%d files deleted", n);
But this does not solve the problem. It helps languages where the plural form of a noun is not simply constructed by adding an `s' but that is all. Once again people fell into the trap of believing the rules their language is using are universal. But the handling of plural forms differs widely between the language families. There are two things we can differ between (and even inside language families);
But other language families have only one form or many forms. More information on this in an extra section.
The consequence of this is that application writers should not try to
solve the problem in their code. This would be localization since it is
only usable for certain, hardcoded language environments. Instead the
extended gettext
interface should be used.
These extra functions are taking instead of the one key string two
strings and an numerical argument. The idea behind this is that using
the numerical argument and the first string as a key, the implementation
can select using rules specified by the translator the right plural
form. The two string arguments then will be used to provide a return
value in case no message catalog is found (similar to the normal
gettext
behavior). In this case the rules for Germanic language
is used and it is assumed that the first string argument is the singular
form, the second the plural form.
This has the consequence that programs without language catalogs can
display the correct strings only if the program itself is written using
a Germanic language. This is a limitation but since the GNU C library
(as well as the GNU gettext
package) are written as part of the
GNU package and the coding standards for the GNU project require program
being written in English, this solution nevertheless fulfills its
purpose.
char * ngettext (const char *msgid1, const char *msgid2, unsigned long int n) | Function |
The ngettext function is similar to the gettext function
as it finds the message catalogs in the same way. But it takes two
extra arguments. The msgid1 parameter must contain the singular
form of the string to be converted. It is also used as the key for the
search in the catalog. The msgid2 parameter is the plural form.
The parameter n is used to determine the plural form. If no
message catalog is found msgid1 is returned if n == 1 ,
otherwise msgid2 .
An example for the us of this function is:
printf (ngettext ("%d file removed", "%d files removed", n), n); Please note that the numeric value n has to be passed to the
|
char * dngettext (const char *domain, const char *msgid1, const char *msgid2, unsigned long int n) | Function |
The dngettext is similar to the dgettext function in the
way the message catalog is selected. The difference is that it takes
two extra parameter to provide the correct plural form. These two
parameters are handled in the same way ngettext handles them.
|
char * dcngettext (const char *domain, const char *msgid1, const char *msgid2, unsigned long int n, int category) | Function |
The dcngettext is similar to the dcgettext function in the
way the message catalog is selected. The difference is that it takes
two extra parameter to provide the correct plural form. These two
parameters are handled in the same way ngettext handles them.
|
A description of the problem can be found at the beginning of the last section. Now there is the question how to solve it. Without the input of linguists (which was not available) it was not possible to determine whether there are only a few different forms in which plural forms are formed or whether the number can increase with every new supported language.
Therefore the solution implemented is to allow the translator to specify
the rules of how to select the plural form. Since the formula varies
with every language this is the only viable solution except for
hardcoding the information in the code (which still would require the
possibility of extensions to not prevent the use of new languages). The
details are explained in the GNU gettext
manual. Here only a a
bit of information is provided.
The information about the plural form selection has to be stored in the
header entry (the one with the empty (msgid
string). It looks
like this:
Plural-Forms: nplurals=2; plural=n == 1 ? 0 : 1;
The nplurals
value must be a decimal number which specifies how
many different plural forms exist for this language. The string
following plural
is an expression which is using the C language
syntax. Exceptions are that no negative number are allowed, numbers
must be decimal, and the only variable allowed is n
. This
expression will be evaluated whenever one of the functions
ngettext
, dngettext
, or dcngettext
is called. The
numeric value passed to these functions is then substituted for all uses
of the variable n
in the expression. The resulting value then
must be greater or equal to zero and smaller than the value given as the
value of nplurals
.
The following rules are known at this point. The language with families are listed. But this does not necessarily mean the information can be generalized for the whole family (as can be easily seen in the table below).1
Plural-Forms: nplurals=1; plural=0;
Languages with this property include:
Plural-Forms: nplurals=2; plural=n != 1;
(Note: this uses the feature of C expressions that boolean expressions have to value zero or one.)
Languages with this property include:
Plural-Forms: nplurals=2; plural=n>1;
Languages with this property include:
Plural-Forms: nplurals=3; plural=n==1 ? 0 : n==2 ? 1 : 2;
Languages with this property include:
Plural-Forms: nplurals=3; \ plural=n%100/10==1 ? 2 : n%10==1 ? 0 : (n+9)%10>3 ? 2 : 1;
Languages with this property include:
Plural-Forms: nplurals=3; \ plural=(n==1) ? 1 : (n>=2 && n<=4) ? 2 : 0;
Languages with this property include:
Plural-Forms: nplurals=3; \ plural=n==1 ? 0 : \ n%10>=2 && n%10<=4 && (n%100<10 || n%100>=20) ? 1 : 2;
(Continuation in the next line is possible.)
Languages with this property include:
Plural-Forms: nplurals=4; \ plural=n==1 ? 0 : n%10==2 ? 1 : n%10==3 || n%10==4 ? 2 : 3;
Languages with this property include:
gettext
usesgettext
not only looks up a translation in a message catalog. It
also converts the translation on the fly to the desired output character
set. This is useful if the user is working in a different character set
than the translator who created the message catalog, because it avoids
distributing variants of message catalogs which differ only in the
character set.
The output character set is, by default, the value of nl_langinfo
(CODESET)
, which depends on the LC_CTYPE
part of the current
locale. But programs which store strings in a locale independent way
(e.g. UTF-8) can request that gettext
and related functions
return the translations in that encoding, by use of the
bind_textdomain_codeset
function.
Note that the msgid argument to gettext
is not subject to
character set conversion. Also, when gettext
does not find a
translation for msgid, it returns msgid unchanged -
independently of the current output character set. It is therefore
recommended that all msgids be US-ASCII strings.
char * bind_textdomain_codeset (const char *domainname, const char *codeset) | Function |
The bind_textdomain_codeset function can be used to specify the
output character set for message catalogs for domain domainname.
The codeset argument must be a valid codeset name which can be used
for the iconv_open function, or a null pointer.
If the codeset parameter is the null pointer,
The The |
gettext
in GUI programsOne place where the gettext
functions, if used normally, have big
problems is within programs with graphical user interfaces (GUIs). The
problem is that many of the strings which have to be translated are very
short. They have to appear in pull-down menus which restricts the
length. But strings which are not containing entire sentences or at
least large fragments of a sentence may appear in more than one
situation in the program but might have different translations. This is
especially true for the one-word strings which are frequently used in
GUI programs.
As a consequence many people say that the gettext
approach is
wrong and instead catgets
should be used which indeed does not
have this problem. But there is a very simple and powerful method to
handle these kind of problems with the gettext
functions.
As as example consider the following fictional situation. A GUI program
has a menu bar with the following entries:
+------------+------------+--------------------------------------+ | File | Printer | | +------------+------------+--------------------------------------+ | Open | | Select | | New | | Open | +----------+ | Connect | +----------+
To have the strings File
, Printer
, Open
,
New
, Select
, and Connect
translated there has to be
at some point in the code a call to a function of the gettext
family. But in two places the string passed into the function would be
Open
. The translations might not be the same and therefore we
are in the dilemma described above.
One solution to this problem is to artificially enlengthen the strings to make them unambiguous. But what would the program do if no translation is available? The enlengthened string is not what should be printed. So we should use a little bit modified version of the functions.
To enlengthen the strings a uniform method should be used. E.g., in the
example above the strings could be chosen as
Menu|File Menu|Printer Menu|File|Open Menu|File|New Menu|Printer|Select Menu|Printer|Open Menu|Printer|Connect
Now all the strings are different and if now instead of gettext
the following little wrapper function is used, everything works just
fine:
char * sgettext (const char *msgid) { char *msgval = gettext (msgid); if (msgval == msgid) msgval = strrchr (msgid, '|') + 1; return msgval; }
What this little function does is to recognize the case when no
translation is available. This can be done very efficiently by a
pointer comparison since the return value is the input value. If there
is no translation we know that the input string is in the format we used
for the Menu entries and therefore contains a |
character. We
simply search for the last occurrence of this character and return a
pointer to the character following it. That's it!
If one now consistently uses the enlengthened string form and replaces
the gettext
calls with calls to sgettext
(this is normally
limited to very few places in the GUI implementation) then it is
possible to produce a program which can be internationalized.
With advanced compilers (such as GNU C) one can write the
sgettext
functions as an inline function or as a macro like this:
#define sgettext(msgid) \ ({ const char *__msgid = (msgid); \ char *__msgstr = gettext (__msgid); \ if (__msgval == __msgid) \ __msgval = strrchr (__msgid, '|') + 1; \ __msgval; })
The other gettext
functions (dgettext
, dcgettext
and the ngettext
equivalents) can and should have corresponding
functions as well which look almost identical, except for the parameters
and the call to the underlying function.
Now there is of course the question why such functions do not exist in the GNU C library? There are two parts of the answer to this question.
|
which is a quite good choice because it
resembles a notation frequently used in this context and it also is a
character not often used in message strings.
But what if the character is used in message strings. Or if the chose
character is not available in the character set on the machine one
compiles (e.g., |
is not required to exist for ISO C; this is
why the iso646.h
file exists in ISO C programming environments).
There is only one more comment to make left. The wrapper function above require that the translations strings are not enlengthened themselves. This is only logical. There is no need to disambiguate the strings (since they are never used as keys for a search) and one also saves quite some memory and disk space by doing this.
gettext
The last sections described what the programmer can do to internationalize the messages of the program. But it is finally up to the user to select the message s/he wants to see. S/He must understand them.
The POSIX locale model uses the environment variables LC_COLLATE
,
LC_CTYPE
, LC_MESSAGES
, LC_MONETARY
, NUMERIC
,
and LC_TIME
to select the locale which is to be used. This way
the user can influence lots of functions. As we mentioned above the
gettext
functions also take advantage of this.
To understand how this happens it is necessary to take a look at the
various components of the filename which gets computed to locate a
message catalog. It is composed as follows:
dir_name/locale/LC_category/domain_name.mo
The default value for dir_name is system specific. It is computed
from the value given as the prefix while configuring the C library.
This value normally is /usr
or /
. For the former the
complete dir_name is:
/usr/share/locale
We can use /usr/share
since the .mo
files containing the
message catalogs are system independent, so all systems can use the same
files. If the program executed the bindtextdomain
function for
the message domain that is currently handled, the dir_name
component is exactly the value which was given to the function as
the second parameter. I.e., bindtextdomain
allows overwriting
the only system dependent and fixed value to make it possible to
address files anywhere in the filesystem.
The category is the name of the locale category which was selected
in the program code. For gettext
and dgettext
this is
always LC_MESSAGES
, for dcgettext
this is selected by the
value of the third parameter. As said above it should be avoided to
ever use a category other than LC_MESSAGES
.
The locale component is computed based on the category used. Just
like for the setlocale
function here comes the user selection
into the play. Some environment variables are examined in a fixed order
and the first environment variable set determines the return value of
the lookup process. In detail, for the category LC_xxx
the
following variables in this order are examined:
LANGUAGE
LC_ALL
LC_xxx
LANG
This looks very familiar. With the exception of the LANGUAGE
environment variable this is exactly the lookup order the
setlocale
function uses. But why introducing the LANGUAGE
variable?
The reason is that the syntax of the values these variables can have is
different to what is expected by the setlocale
function. If we
would set LC_ALL
to a value following the extended syntax that
would mean the setlocale
function will never be able to use the
value of this variable as well. An additional variable removes this
problem plus we can select the language independently of the locale
setting which sometimes is useful.
While for the LC_xxx
variables the value should consist of
exactly one specification of a locale the LANGUAGE
variable's
value can consist of a colon separated list of locale names. The
attentive reader will realize that this is the way we manage to
implement one of our additional demands above: we want to be able to
specify an ordered list of language.
Back to the constructed filename we have only one component missing.
The domain_name part is the name which was either registered using
the textdomain
function or which was given to dgettext
or
dcgettext
as the first parameter. Now it becomes obvious that a
good choice for the domain name in the program code is a string which is
closely related to the program/package name. E.g., for the GNU C
Library the domain name is libc
.
A limit piece of example code should show how the programmer is supposed
to work:
{ setlocale (LC_ALL, ""); textdomain ("test-package"); bindtextdomain ("test-package", "/usr/local/share/locale"); puts (gettext ("Hello, world!")); }
At the program start the default domain is messages
, and the
default locale is "C". The setlocale
call sets the locale
according to the user's environment variables; remember that correct
functioning of gettext
relies on the correct setting of the
LC_MESSAGES
locale (for looking up the message catalog) and
of the LC_CTYPE
locale (for the character set conversion).
The textdomain
call changes the default domain to
test-package
. The bindtextdomain
call specifies that
the message catalogs for the domain test-package
can be found
below the directory /usr/local/share/locale
.
If now the user set in her/his environment the variable LANGUAGE
to de
the gettext
function will try to use the
translations from the file
/usr/local/share/locale/de/LC_MESSAGES/test-package.mo
From the above descriptions it should be clear which component of this filename is determined by which source.
In the above example we assumed that the LANGUAGE
environment
variable to de
. This might be an appropriate selection but what
happens if the user wants to use LC_ALL
because of the wider
usability and here the required value is de_DE.ISO-8859-1
? We
already mentioned above that a situation like this is not infrequent.
E.g., a person might prefer reading a dialect and if this is not
available fall back on the standard language.
The gettext
functions know about situations like this and can
handle them gracefully. The functions recognize the format of the value
of the environment variable. It can split the value is different pieces
and by leaving out the only or the other part it can construct new
values. This happens of course in a predictable way. To understand
this one must know the format of the environment variable value. There
are two more or less standardized forms:
language[_territory[.codeset]][@modifier]
language[_territory][+audience][+special][,[sponsor][_revision]]
The functions will automatically recognize which format is used. Less specific locale names will be stripped of in the order of the following list:
revision
sponsor
special
codeset
normalized codeset
territory
audience
/modifier
From the last entry one can see that the meaning of the modifier
field in the X/Open format and the audience
format have the same
meaning. Beside one can see that the language
field for obvious
reasons never will be dropped.
The only new thing is the normalized codeset
entry. This is
another goodie which is introduced to help reducing the chaos which
derives from the inability of the people to standardize the names of
character sets. Instead of ISO-8859-1 one can often see 8859-1,
88591, iso8859-1, or iso_8859-1. The normalized
codeset
value is generated from the user-provided character set name by
applying the following rules:
"iso"
.
So all of the above name will be normalized to iso88591
. This
allows the program user much more freely choosing the locale name.
Even this extended functionality still does not help to solve the
problem that completely different names can be used to denote the same
locale (e.g., de
and german
). To be of help in this
situation the locale implementation and also the gettext
functions know about aliases.
The file /usr/share/locale/locale.alias
(replace /usr
with
whatever prefix you used for configuring the C library) contains a
mapping of alternative names to more regular names. The system manager
is free to add new entries to fill her/his own needs. The selected
locale from the environment is compared with the entries in the first
column of this file ignoring the case. If they match the value of the
second column is used instead for the further handling.
In the description of the format of the environment variables we already mentioned the character set as a factor in the selection of the message catalog. In fact, only catalogs which contain text written using the character set of the system/program can be used (directly; there will come a solution for this some day). This means for the user that s/he will always have to take care for this. If in the collection of the message catalogs there are files for the same language but coded using different character sets the user has to be careful.
gettext
The GNU C Library does not contain the source code for the programs to
handle message catalogs for the gettext
functions. As part of
the GNU project the GNU gettext package contains everything the
developer needs. The functionality provided by the tools in this
package by far exceeds the abilities of the gencat
program
described above for the catgets
functions.
There is a program msgfmt
which is the equivalent program to the
gencat
program. It generates from the human-readable and
-editable form of the message catalog a binary file which can be used by
the gettext
functions. But there are several more programs
available.
The xgettext
program can be used to automatically extract the
translatable messages from a source file. I.e., the programmer need not
take care for the translations and the list of messages which have to be
translated. S/He will simply wrap the translatable string in calls to
gettext
et.al and the rest will be done by xgettext
. This
program has a lot of option which help to customize the output or do
help to understand the input better.
Other programs help to manage development cycle when new messages appear in the source files or when a new translation of the messages appear. here it should only be noted that using all the tools in GNU gettext it is possible to completely automize the handling of message catalog. Beside marking the translatable string in the source code and generating the translations the developers do not have anything to do themselves.
This chapter describes functions for searching and sorting arrays of arbitrary objects. You pass the appropriate comparison function to be applied as an argument, along with the size of the objects in the array and the total number of elements.
bsearch
function.
qsort
function.
hsearch
function.
tsearch
function.
In order to use the sorted array library functions, you have to describe how to compare the elements of the array.
To do this, you supply a comparison function to compare two elements of
the array. The library will call this function, passing as arguments
pointers to two array elements to be compared. Your comparison function
should return a value the way strcmp
(see String/Array Comparison) does: negative if the first argument is "less" than the
second, zero if they are "equal", and positive if the first argument
is "greater".
Here is an example of a comparison function which works with an array of
numbers of type double
:
int compare_doubles (const void *a, const void *b) { const double *da = (const double *) a; const double *db = (const double *) b; return (*da > *db) - (*da < *db); }
The header file stdlib.h
defines a name for the data type of
comparison functions. This type is a GNU extension.
int comparison_fn_t (const void *, const void *);
Generally searching for a specific element in an array means that
potentially all elements must be checked. The GNU C library contains
functions to perform linear search. The prototypes for the following
two functions can be found in search.h
.
void * lfind (const void *key, void *base, size_t *nmemb, size_t size, comparison_fn_t compar) | Function |
The lfind function searches in the array with *nmemb
elements of size bytes pointed to by base for an element
which matches the one pointed to by key. The function pointed to
by compar is used decide whether two elements match.
The return value is a pointer to the matching element in the array
starting at base if it is found. If no matching element is
available The mean runtime of this function is |
void * lsearch (const void *key, void *base, size_t *nmemb, size_t size, comparison_fn_t compar) | Function |
The lsearch function is similar to the lfind function. It
searches the given array for an element and returns it if found. The
difference is that if no matching element is found the lsearch
function adds the object pointed to by key (with a size of
size bytes) at the end of the array and it increments the value of
*nmemb to reflect this addition.
This means for the caller that if it is not sure that the array contains
the element one is searching for the memory allocated for the array
starting at base must have room for at least size more
bytes. If one is sure the element is in the array it is better to use
|
To search a sorted array for an element matching the key, use the
bsearch
function. The prototype for this function is in
the header file stdlib.h
.
void * bsearch (const void *key, const void *array, size_t count, size_t size, comparison_fn_t compare) | Function |
The bsearch function searches the sorted array array for an object
that is equivalent to key. The array contains count elements,
each of which is of size size bytes.
The compare function is used to perform the comparison. This function is called with two pointer arguments and should return an integer less than, equal to, or greater than zero corresponding to whether its first argument is considered less than, equal to, or greater than its second argument. The elements of the array must already be sorted in ascending order according to this comparison function. The return value is a pointer to the matching array element, or a null pointer if no match is found. If the array contains more than one element that matches, the one that is returned is unspecified. This function derives its name from the fact that it is implemented using the binary search algorithm. |
To sort an array using an arbitrary comparison function, use the
qsort
function. The prototype for this function is in
stdlib.h
.
void qsort (void *array, size_t count, size_t size, comparison_fn_t compare) | Function |
The qsort function sorts the array array. The array contains
count elements, each of which is of size size.
The compare function is used to perform the comparison on the array elements. This function is called with two pointer arguments and should return an integer less than, equal to, or greater than zero corresponding to whether its first argument is considered less than, equal to, or greater than its second argument. Warning: If two objects compare as equal, their order after sorting is unpredictable. That is to say, the sorting is not stable. This can make a difference when the comparison considers only part of the elements. Two elements with the same sort key may differ in other respects. If you want the effect of a stable sort, you can get this result by writing the comparison function so that, lacking other reason distinguish between two elements, it compares them by their addresses. Note that doing this may make the sorting algorithm less efficient, so do it only if necessary. Here is a simple example of sorting an array of doubles in numerical
order, using the comparison function defined above (see Comparison Functions):
{ double *array; int size; ... qsort (array, size, sizeof (double), compare_doubles); } The The implementation of |
Here is an example showing the use of qsort
and bsearch
with an array of structures. The objects in the array are sorted
by comparing their name
fields with the strcmp
function.
Then, we can look up individual objects based on their names.
#include <stdlib.h> #include <stdio.h> #include <string.h> /* Define an array of critters to sort. */ struct critter { const char *name; const char *species; }; struct critter muppets[] = { {"Kermit", "frog"}, {"Piggy", "pig"}, {"Gonzo", "whatever"}, {"Fozzie", "bear"}, {"Sam", "eagle"}, {"Robin", "frog"}, {"Animal", "animal"}, {"Camilla", "chicken"}, {"Sweetums", "monster"}, {"Dr. Strangepork", "pig"}, {"Link Hogthrob", "pig"}, {"Zoot", "human"}, {"Dr. Bunsen Honeydew", "human"}, {"Beaker", "human"}, {"Swedish Chef", "human"} }; int count = sizeof (muppets) / sizeof (struct critter); /* This is the comparison function used for sorting and searching. */ int critter_cmp (const struct critter *c1, const struct critter *c2) { return strcmp (c1->name, c2->name); } /* Print information about a critter. */ void print_critter (const struct critter *c) { printf ("%s, the %s\n", c->name, c->species); } /* Do the lookup into the sorted array. */ void find_critter (const char *name) { struct critter target, *result; target.name = name; result = bsearch (&target, muppets, count, sizeof (struct critter), critter_cmp); if (result) print_critter (result); else printf ("Couldn't find %s.\n", name); } /* Main program. */ int main (void) { int i; for (i = 0; i < count; i++) print_critter (&muppets[i]); printf ("\n"); qsort (muppets, count, sizeof (struct critter), critter_cmp); for (i = 0; i < count; i++) print_critter (&muppets[i]); printf ("\n"); find_critter ("Kermit"); find_critter ("Gonzo"); find_critter ("Janice"); return 0; }
The output from this program looks like:
Kermit, the frog Piggy, the pig Gonzo, the whatever Fozzie, the bear Sam, the eagle Robin, the frog Animal, the animal Camilla, the chicken Sweetums, the monster Dr. Strangepork, the pig Link Hogthrob, the pig Zoot, the human Dr. Bunsen Honeydew, the human Beaker, the human Swedish Chef, the human Animal, the animal Beaker, the human Camilla, the chicken Dr. Bunsen Honeydew, the human Dr. Strangepork, the pig Fozzie, the bear Gonzo, the whatever Kermit, the frog Link Hogthrob, the pig Piggy, the pig Robin, the frog Sam, the eagle Swedish Chef, the human Sweetums, the monster Zoot, the human Kermit, the frog Gonzo, the whatever Couldn't find Janice.
hsearch
function.The functions mentioned so far in this chapter are searching in a sorted or unsorted array. There are other methods to organize information which later should be searched. The costs of insert, delete and search differ. One possible implementation is using hashing tables.
int hcreate (size_t nel) | Function |
The hcreate function creates a hashing table which can contain at
least nel elements. There is no possibility to grow this table so
it is necessary to choose the value for nel wisely. The used
methods to implement this function might make it necessary to make the
number of elements in the hashing table larger than the expected maximal
number of elements. Hashing tables usually work inefficient if they are
filled 80% or more. The constant access time guaranteed by hashing can
only be achieved if few collisions exist. See Knuth's "The Art of
Computer Programming, Part 3: Searching and Sorting" for more
information.
The weakest aspect of this function is that there can be at most one hashing table used through the whole program. The table is allocated in local memory out of control of the programmer. As an extension the GNU C library provides an additional set of functions with an reentrant interface which provide a similar interface but which allow to keep arbitrarily many hashing tables. It is possible to use more than one hashing table in the program run if
the former table is first destroyed by a call to The function returns a non-zero value if successful. If it return zero something went wrong. This could either mean there is already a hashing table in use or the program runs out of memory. |
void hdestroy (void) | Function |
The hdestroy function can be used to free all the resources
allocated in a previous call of hcreate . After a call to this
function it is again possible to call hcreate and allocate a new
table with possibly different size.
It is important to remember that the elements contained in the hashing
table at the time |
Entries of the hashing table and keys for the search are defined using this type:
struct ENTRY | Data type |
Both elements of this structure are pointers to zero-terminated strings.
This is a limiting restriction of the functionality of the
hsearch functions. They can only be used for data sets which use
the NUL character always and solely to terminate the records. It is not
possible to handle general binary data.
|
ENTRY * hsearch (ENTRY item, ACTION action) | Function |
To search in a hashing table created using hcreate the
hsearch function must be used. This function can perform simple
search for an element (if action has the FIND ) or it can
alternatively insert the key element into the hashing table, possibly
replacing a previous value (if action is ENTER ).
The key is denoted by a pointer to an object of type The return value depends on the action parameter value. If it is
|
As mentioned before the hashing table used by the functions described so
far is global and there can be at any time at most one hashing table in
the program. A solution is to use the following functions which are a
GNU extension. All have in common that they operate on a hashing table
which is described by the content of an object of the type struct
hsearch_data
. This type should be treated as opaque, none of its
members should be changed directly.
int hcreate_r (size_t nel, struct hsearch_data *htab) | Function |
The hcreate_r function initializes the object pointed to by
htab to contain a hashing table with at least nel elements.
So this function is equivalent to the hcreate function except
that the initialized data structure is controlled by the user.
This allows having more than one hashing table at one time. The
memory necessary for the The return value is non-zero if the operation were successful. if the return value is zero something went wrong which probably means the programs runs out of memory. |
void hdestroy_r (struct hsearch_data *htab) | Function |
The hdestroy_r function frees all resources allocated by the
hcreate_r function for this very same object htab. As for
hdestroy it is the programs responsibility to free the strings
for the elements of the table.
|
int hsearch_r (ENTRY item, ACTION action, ENTRY **retval, struct hsearch_data *htab) | Function |
The hsearch_r function is equivalent to hsearch . The
meaning of the first two arguments is identical. But instead of
operating on a single global hashing table the function works on the
table described by the object pointed to by htab (which is
initialized by a call to hcreate_r ).
Another difference to
|
tsearch
function.Another common form to organize data for efficient search is to use
trees. The tsearch
function family provides a nice interface to
functions to organize possibly large amounts of data by providing a mean
access time proportional to the logarithm of the number of elements.
The GNU C library implementation even guarantees that this bound is
never exceeded even for input data which cause problems for simple
binary tree implementations.
The functions described in the chapter are all described in the System V and X/Open specifications and are therefore quite portable.
In contrast to the hsearch
functions the tsearch
functions
can be used with arbitrary data and not only zero-terminated strings.
The tsearch
functions have the advantage that no function to
initialize data structures is necessary. A simple pointer of type
void *
initialized to NULL
is a valid tree and can be
extended or searched.
void * tsearch (const void *key, void **rootp, comparison_fn_t compar) | Function |
The tsearch function searches in the tree pointed to by
*rootp for an element matching key. The function
pointed to by compar is used to determine whether two elements
match. See Comparison Functions, for a specification of the functions
which can be used for the compar parameter.
If the tree does not contain a matching entry the key value will
be added to the tree. The tree is represented by a pointer to a pointer since it is sometimes
necessary to change the root node of the tree. So it must not be
assumed that the variable pointed to by rootp has the same value
after the call. This also shows that it is not safe to call the
The return value is a pointer to the matching element in the tree. If a
new element was created the pointer points to the new data (which is in
fact key). If an entry had to be created and the program ran out
of space |
void * tfind (const void *key, void *const *rootp, comparison_fn_t compar) | Function |
The tfind function is similar to the tsearch function. It
locates an element matching the one pointed to by key and returns
a pointer to this element. But if no matching element is available no
new element is entered (note that the rootp parameter points to a
constant pointer). Instead the function returns NULL .
|
Another advantage of the tsearch
function in contrast to the
hsearch
functions is that there is an easy way to remove
elements.
void * tdelete (const void *key, void **rootp, comparison_fn_t compar) | Function |
To remove a specific element matching key from the tree
tdelete can be used. It locates the matching element using the
same method as tfind . The corresponding element is then removed
and a pointer to the parent of the deleted node is returned by the
function. If there is no matching entry in the tree nothing can be
deleted and the function returns NULL . If the root of the tree
is deleted tdelete returns some unspecified value not equal to
NULL .
|
void tdestroy (void *vroot, __free_fn_t freefct) | Function |
If the complete search tree has to be removed one can use
tdestroy . It frees all resources allocated by the tsearch
function to generate the tree pointed to by vroot.
For the data in each tree node the function freefct is called. The pointer to the data is passed as the argument to the function. If no such work is necessary freefct must point to a function doing nothing. It is called in any case. This function is a GNU extension and not covered by the System V or X/Open specifications. |
In addition to the function to create and destroy the tree data
structure, there is another function which allows you to apply a
function to all elements of the tree. The function must have this type:
void __action_fn_t (const void *nodep, VISIT value, int level);
The nodep is the data value of the current node (once given as the
key argument to tsearch
). level is a numeric value
which corresponds to the depth of the current node in the tree. The
root node has the depth 0 and its children have a depth of
1 and so on. The VISIT
type is an enumeration type.
VISIT | Data Type |
The VISIT value indicates the status of the current node in the
tree and how the function is called. The status of a node is either
`leaf' or `internal node'. For each leaf node the function is called
exactly once, for each internal node it is called three times: before
the first child is processed, after the first child is processed and
after both children are processed. This makes it possible to handle all
three methods of tree traversal (or even a combination of them).
|
void twalk (const void *root, __action_fn_t action) | Function |
For each node in the tree with a node pointed to by root, the
twalk function calls the function provided by the parameter
action. For leaf nodes the function is called exactly once with
value set to leaf . For internal nodes the function is
called three times, setting the value parameter or action to
the appropriate value. The level argument for the action
function is computed while descending the tree with increasing the value
by one for the descend to a child, starting with the value 0 for
the root node.
Since the functions used for the action parameter to |
The GNU C Library provides pattern matching facilities for two kinds of patterns: regular expressions and file-name wildcards. The library also provides a facility for expanding variable and command references and parsing text into words in the way the shell does.
This section describes how to match a wildcard pattern against a
particular string. The result is a yes or no answer: does the
string fit the pattern or not. The symbols described here are all
declared in fnmatch.h
.
int fnmatch (const char *pattern, const char *string, int flags) | Function |
This function tests whether the string string matches the pattern
pattern. It returns 0 if they do match; otherwise, it
returns the nonzero value FNM_NOMATCH . The arguments
pattern and string are both strings.
The argument flags is a combination of flag bits that alter the details of matching. See below for a list of the defined flags. In the GNU C Library, |
These are the available flags for the flags argument:
FNM_FILE_NAME
/
character specially, for matching file names. If
this flag is set, wildcard constructs in pattern cannot match
/
in string. Thus, the only way to match /
is with
an explicit /
in pattern.
FNM_PATHNAME
FNM_FILE_NAME
; it comes from POSIX.2. We
don't recommend this name because we don't use the term "pathname" for
file names.
FNM_PERIOD
.
character specially if it appears at the beginning of
string. If this flag is set, wildcard constructs in pattern
cannot match .
as the first character of string.
If you set both FNM_PERIOD
and FNM_FILE_NAME
, then the
special treatment applies to .
following /
as well as to
.
at the beginning of string. (The shell uses the
FNM_PERIOD
and FNM_FILE_NAME
flags together for matching
file names.)
FNM_NOESCAPE
\
character specially in patterns. Normally,
\
quotes the following character, turning off its special meaning
(if any) so that it matches only itself. When quoting is enabled, the
pattern \?
matches only the string ?
, because the question
mark in the pattern acts like an ordinary character.
If you use FNM_NOESCAPE
, then \
is an ordinary character.
FNM_LEADING_DIR
/
in
string; that is to say, test whether string starts with a
directory name that pattern matches.
If this flag is set, either foo*
or foobar
as a pattern
would match the string foobar/frobozz
.
FNM_CASEFOLD
FNM_EXTMATCH
ksh
. The patterns are written in the form
explained in the following table where pattern-list is a |
separated list of patterns.
?(pattern-list)
*(pattern-list)
+(pattern-list)
@(pattern-list)
!(pattern-list)
The archetypal use of wildcards is for matching against the files in a directory, and making a list of all the matches. This is called globbing.
You could do this using fnmatch
, by reading the directory entries
one by one and testing each one with fnmatch
. But that would be
slow (and complex, since you would have to handle subdirectories by
hand).
The library provides a function glob
to make this particular use
of wildcards convenient. glob
and the other symbols in this
section are declared in glob.h
.
glob
.
glob
.
glob
.
glob
The result of globbing is a vector of file names (strings). To return
this vector, glob
uses a special data type, glob_t
, which
is a structure. You pass glob
the address of the structure, and
it fills in the structure's fields to tell you about the results.
glob_t | Data Type |
This data type holds a pointer to a word vector. More precisely, it
records both the address of the word vector and its size. The GNU
implementation contains some more fields which are non-standard
extensions.
|
For use in the glob64
function glob.h
contains another
definition for a very similar type. glob64_t
differs from
glob_t
only in the types of the members gl_readdir
,
gl_stat
, and gl_lstat
.
glob64_t | Data Type |
This data type holds a pointer to a word vector. More precisely, it
records both the address of the word vector and its size. The GNU
implementation contains some more fields which are non-standard
extensions.
|
int glob (const char *pattern, int flags, int (*errfunc) (const char *filename, int error-code), glob_t *vector-ptr) | Function |
The function glob does globbing using the pattern pattern
in the current directory. It puts the result in a newly allocated
vector, and stores the size and address of this vector into
*vector-ptr . The argument flags is a combination of
bit flags; see Flags for Globbing, for details of the flags.
The result of globbing is a sequence of file names. The function
To return this vector, Normally, If
In the event of an error, It is important to notice that the |
int glob64 (const char *pattern, int flags, int (*errfunc) (const char *filename, int error-code), glob64_t *vector-ptr) | Function |
The glob64 function was added as part of the Large File Summit
extensions but is not part of the original LFS proposal. The reason for
this is simple: it is not necessary. The necessity for a glob64
function is added by the extensions of the GNU glob
implementation which allows the user to provide own directory handling
and stat functions. The readdir and stat functions
do depend on the choice of _FILE_OFFSET_BITS since the definition
of the types struct dirent and struct stat will change
depending on the choice.
Beside this difference the This function is a GNU extension. |
This section describes the flags that you can specify in the
flags argument to glob
. Choose the flags you want,
and combine them with the C bitwise OR operator |
.
GLOB_APPEND
glob
. This way you can effectively expand
several words as if they were concatenated with spaces between them.
In order for appending to work, you must not modify the contents of the
word vector structure between calls to glob
. And, if you set
GLOB_DOOFFS
in the first call to glob
, you must also
set it when you append to the results.
Note that the pointer stored in gl_pathv
may no longer be valid
after you call glob
the second time, because glob
might
have relocated the vector. So always fetch gl_pathv
from the
glob_t
structure after each glob
call; never save
the pointer across calls.
GLOB_DOOFFS
gl_offs
field says how many slots to leave.
The blank slots contain null pointers.
GLOB_ERR
glob
tries its best to keep
on going despite any errors, reading whatever directories it can.
You can exercise even more control than this by specifying an
error-handler function errfunc when you call glob
. If
errfunc is not a null pointer, then glob
doesn't give up
right away when it can't read a directory; instead, it calls
errfunc with two arguments, like this:
(*errfunc) (filename, error-code)
The argument filename is the name of the directory that
glob
couldn't open or couldn't read, and error-code is the
errno
value that was reported to glob
.
If the error handler function returns nonzero, then glob
gives up
right away. Otherwise, it continues.
GLOB_MARK
/
to the
directory's name when returning it.
GLOB_NOCHECK
glob
returns that there were no
matches.)
GLOB_NOSORT
GLOB_NOESCAPE
\
character specially in patterns. Normally,
\
quotes the following character, turning off its special meaning
(if any) so that it matches only itself. When quoting is enabled, the
pattern \?
matches only the string ?
, because the question
mark in the pattern acts like an ordinary character.
If you use GLOB_NOESCAPE
, then \
is an ordinary character.
glob
does its work by calling the function fnmatch
repeatedly. It handles the flag GLOB_NOESCAPE
by turning on the
FNM_NOESCAPE
flag in calls to fnmatch
.
Beside the flags described in the last section, the GNU implementation of
glob
allows a few more flags which are also defined in the
glob.h
file. Some of the extensions implement functionality
which is available in modern shell implementations.
GLOB_PERIOD
.
character (period) is treated special. It cannot be
matched by wildcards. See Wildcard Matching, FNM_PERIOD
.
GLOB_MAGCHAR
GLOB_MAGCHAR
value is not to be given to glob
in the
flags parameter. Instead, glob
sets this bit in the
gl_flags element of the glob_t structure provided as the
result if the pattern used for matching contains any wildcard character.
GLOB_ALTDIRFUNC
glob
implementation uses the user-supplied
functions specified in the structure pointed to by pglob
parameter. For more information about the functions refer to the
sections about directory handling see Accessing Directories, and
Reading Attributes.
GLOB_BRACE
The string between the matching braces is separated into single
expressions by splitting at ,
(comma) characters. The commas
themselves are discarded. Please note what we said above about recursive
brace expressions. The commas used to separate the subexpressions must
be at the same level. Commas in brace subexpressions are not matched.
They are used during expansion of the brace expression of the deeper
level. The example below shows this
glob ("{foo/{,bar,biz},baz}", GLOB_BRACE, NULL, &result)
is equivalent to the sequence
glob ("foo/", GLOB_BRACE, NULL, &result) glob ("foo/bar", GLOB_BRACE|GLOB_APPEND, NULL, &result) glob ("foo/biz", GLOB_BRACE|GLOB_APPEND, NULL, &result) glob ("baz", GLOB_BRACE|GLOB_APPEND, NULL, &result)
if we leave aside error handling.
GLOB_NOMAGIC
GLOB_TILDE
~
(tilde) is handled special
if it appears at the beginning of the pattern. Instead of being taken
verbatim it is used to represent the home directory of a known user.
If ~
is the only character in pattern or it is followed by a
/
(slash), the home directory of the process owner is
substituted. Using getlogin
and getpwnam
the information
is read from the system databases. As an example take user bart
with his home directory at /home/bart
. For him a call like
glob ("~/bin/*", GLOB_TILDE, NULL, &result)
would return the contents of the directory /home/bart/bin
.
Instead of referring to the own home directory it is also possible to
name the home directory of other users. To do so one has to append the
user name after the tilde character. So the contents of user
homer
's bin
directory can be retrieved by
glob ("~homer/bin/*", GLOB_TILDE, NULL, &result)
If the user name is not valid or the home directory cannot be determined
for some reason the pattern is left untouched and itself used as the
result. I.e., if in the last example home
is not available the
tilde expansion yields to "~homer/bin/*"
and glob
is not
looking for a directory named ~homer
.
This functionality is equivalent to what is available in C-shells if the
nonomatch
flag is set.
GLOB_TILDE_CHECK
glob
behaves like as if GLOB_TILDE
is
given. The only difference is that if the user name is not available or
the home directory cannot be determined for other reasons this leads to
an error. glob
will return GLOB_NOMATCH
instead of using
the pattern itself as the name.
This functionality is equivalent to what is available in C-shells if
nonomatch
flag is not set.
GLOB_ONLYDIR
This functionality is only available with the GNU glob
implementation. It is mainly used internally to increase the
performance but might be useful for a user as well and therefore is
documented here.
Calling glob
will in most cases allocate resources which are used
to represent the result of the function call. If the same object of
type glob_t
is used in multiple call to glob
the resources
are freed or reused so that no leaks appear. But this does not include
the time when all glob
calls are done.
void globfree (glob_t *pglob) | Function |
The globfree function frees all resources allocated by previous
calls to glob associated with the object pointed to by
pglob. This function should be called whenever the currently used
glob_t typed object isn't used anymore.
|
void globfree64 (glob64_t *pglob) | Function |
This function is equivalent to globfree but it frees records of
type glob64_t which were allocated by glob64 .
|
The GNU C library supports two interfaces for matching regular expressions. One is the standard POSIX.2 interface, and the other is what the GNU system has had for many years.
Both interfaces are declared in the header file regex.h
.
If you define _POSIX_C_SOURCE
, then only the POSIX.2
functions, structures, and constants are declared.
regcomp
to prepare to match.
regcomp
.
regexec
to match the compiled
pattern that you get from regcomp
.
Before you can actually match a regular expression, you must compile it. This is not true compilation--it produces a special data structure, not machine instructions. But it is like ordinary compilation in that its purpose is to enable you to "execute" the pattern fast. (See Matching POSIX Regexps, for how to use the compiled regular expression for matching.)
There is a special data type for compiled regular expressions:
regex_t | Data Type |
This type of object holds a compiled regular expression.
It is actually a structure. It has just one field that your programs
should look at:
There are several other fields, but we don't describe them here, because only the functions in the library should use them. |
After you create a regex_t
object, you can compile a regular
expression into it by calling regcomp
.
int regcomp (regex_t *compiled, const char *pattern, int cflags) | Function |
The function regcomp "compiles" a regular expression into a
data structure that you can use with regexec to match against a
string. The compiled regular expression format is designed for
efficient matching. regcomp stores it into *compiled .
It's up to you to allocate an object of type The argument cflags lets you specify various options that control the syntax and semantics of regular expressions. See Flags for POSIX Regexps. If you use the flag If you don't use
|
Here are the possible nonzero values that regcomp
can return:
REG_BADBR
\{...\}
construct in the regular
expression. A valid \{...\}
construct must contain either
a single number, or two numbers in increasing order separated by a
comma.
REG_BADPAT
REG_BADRPT
?
or *
appeared in a bad
position (with no preceding subexpression to act on).
REG_ECOLLATE
REG_ECTYPE
REG_EESCAPE
\
.
REG_ESUBREG
\digit
construct.
REG_EBRACK
REG_EPAREN
\(
and \)
.
REG_EBRACE
\{
and \}
.
REG_ERANGE
REG_ESPACE
regcomp
ran out of memory.
These are the bit flags that you can use in the cflags operand when
compiling a regular expression with regcomp
.
REG_EXTENDED
REG_ICASE
REG_NOSUB
REG_NEWLINE
$
can match before the newline and ^
can
match after. Also, don't permit .
to match a newline, and don't
permit [^...]
to match a newline.
Otherwise, newline acts like any other ordinary character.
Once you have compiled a regular expression, as described in POSIX Regexp Compilation, you can match it against strings using
regexec
. A match anywhere inside the string counts as success,
unless the regular expression contains anchor characters (^
or
$
).
int regexec (regex_t *compiled, char *string, size_t nmatch, regmatch_t matchptr [], int eflags) | Function |
This function tries to match the compiled regular expression
*compiled against string.
The argument eflags is a word of bit flags that enable various options. If you want to get information about what part of string actually
matched the regular expression or its subexpressions, use the arguments
matchptr and nmatch. Otherwise, pass |
You must match the regular expression with the same set of current locales that were in effect when you compiled the regular expression.
The function regexec
accepts the following flags in the
eflags argument:
REG_NOTBOL
REG_NOTEOL
Here are the possible nonzero values that regexec
can return:
REG_NOMATCH
REG_ESPACE
regexec
ran out of memory.
When regexec
matches parenthetical subexpressions of
pattern, it records which parts of string they match. It
returns that information by storing the offsets into an array whose
elements are structures of type regmatch_t
. The first element of
the array (index 0
) records the part of the string that matched
the entire regular expression. Each other element of the array records
the beginning and end of the part that matched a single parenthetical
subexpression.
regmatch_t | Data Type |
This is the data type of the matcharray array that you pass to
regexec . It contains two structure fields, as follows:
|
regoff_t | Data Type |
regoff_t is an alias for another signed integer type.
The fields of regmatch_t have type regoff_t .
|
The regmatch_t
elements correspond to subexpressions
positionally; the first element (index 1
) records where the first
subexpression matched, the second element records the second
subexpression, and so on. The order of the subexpressions is the order
in which they begin.
When you call regexec
, you specify how long the matchptr
array is, with the nmatch argument. This tells regexec
how
many elements to store. If the actual regular expression has more than
nmatch subexpressions, then you won't get offset information about
the rest of them. But this doesn't alter whether the pattern matches a
particular string or not.
If you don't want regexec
to return any information about where
the subexpressions matched, you can either supply 0
for
nmatch, or use the flag REG_NOSUB
when you compile the
pattern with regcomp
.
Sometimes a subexpression matches a substring of no characters. This
happens when f\(o*\)
matches the string fum
. (It really
matches just the f
.) In this case, both of the offsets identify
the point in the string where the null substring was found. In this
example, the offsets are both 1
.
Sometimes the entire regular expression can match without using some of
its subexpressions at all--for example, when ba\(na\)*
matches the
string ba
, the parenthetical subexpression is not used. When
this happens, regexec
stores -1
in both fields of the
element for that subexpression.
Sometimes matching the entire regular expression can match a particular
subexpression more than once--for example, when ba\(na\)*
matches the string bananana
, the parenthetical subexpression
matches three times. When this happens, regexec
usually stores
the offsets of the last part of the string that matched the
subexpression. In the case of bananana
, these offsets are
6
and 8
.
But the last match is not always the one that is chosen. It's more
accurate to say that the last opportunity to match is the one
that takes precedence. What this means is that when one subexpression
appears within another, then the results reported for the inner
subexpression reflect whatever happened on the last match of the outer
subexpression. For an example, consider \(ba\(na\)*s \)*
matching
the string bananas bas
. The last time the inner expression
actually matches is near the end of the first word. But it is
considered again in the second word, and fails to match there.
regexec
reports nonuse of the "na" subexpression.
Another place where this rule applies is when the regular expression
\(ba\(na\)*s \|nefer\(ti\)* \)*
matches bananas nefertiti
. The "na" subexpression does match
in the first word, but it doesn't match in the second word because the
other alternative is used there. Once again, the second repetition of
the outer subexpression overrides the first, and within that second
repetition, the "na" subexpression is not used. So regexec
reports nonuse of the "na" subexpression.
When you are finished using a compiled regular expression, you can
free the storage it uses by calling regfree
.
void regfree (regex_t *compiled) | Function |
Calling regfree frees all the storage that *compiled
points to. This includes various internal fields of the regex_t
structure that aren't documented in this manual.
|
You should always free the space in a regex_t
structure with
regfree
before using the structure to compile another regular
expression.
When regcomp
or regexec
reports an error, you can use
the function regerror
to turn it into an error message string.
size_t regerror (int errcode, regex_t *compiled, char *buffer, size_t length) | Function |
This function produces an error message string for the error code
errcode, and stores the string in length bytes of memory
starting at buffer. For the compiled argument, supply the
same compiled regular expression structure that regcomp or
regexec was working with when it got the error. Alternatively,
you can supply NULL for compiled; you will still get a
meaningful error message, but it might not be as detailed.
If the error message can't fit in length bytes (including a
terminating null character), then The return value of Here is a function which uses char *get_regerror (int errcode, regex_t *compiled) { size_t length = regerror (errcode, compiled, NULL, 0); char *buffer = xmalloc (length); (void) regerror (errcode, compiled, buffer, length); return buffer; } |
Word expansion means the process of splitting a string into words and substituting for variables, commands, and wildcards just as the shell does.
For example, when you write ls -l foo.c
, this string is split
into three separate words--ls
, -l
and foo.c
.
This is the most basic function of word expansion.
When you write ls *.c
, this can become many words, because
the word *.c
can be replaced with any number of file names.
This is called wildcard expansion, and it is also a part of
word expansion.
When you use echo $PATH
to print your path, you are taking
advantage of variable substitution, which is also part of word
expansion.
Ordinary programs can perform word expansion just like the shell by
calling the library function wordexp
.
wordexp
.
wordexp
.
When word expansion is applied to a sequence of words, it performs the following transformations in the order shown here:
~foo
with the name of
the home directory of foo
.
$foo
.
`cat foo`
and
the equivalent $(cat foo)
are replaced with the output from
the inner command.
$(($x-1))
are
replaced with the result of the arithmetic computation.
*.c
with a list of .c
file names. Wildcard expansion applies to an
entire word at a time, and replaces that word with 0 or more file names
that are themselves words.
For the details of these transformations, and how to write the constructs that use them, see The BASH Manual (to appear).
wordexp
All the functions, constants and data types for word expansion are
declared in the header file wordexp.h
.
Word expansion produces a vector of words (strings). To return this
vector, wordexp
uses a special data type, wordexp_t
, which
is a structure. You pass wordexp
the address of the structure,
and it fills in the structure's fields to tell you about the results.
wordexp_t | Data Type |
This data type holds a pointer to a word vector. More precisely, it
records both the address of the word vector and its size.
|
int wordexp (const char *words, wordexp_t *word-vector-ptr, int flags) | Function |
Perform word expansion on the string words, putting the result in
a newly allocated vector, and store the size and address of this vector
into *word-vector-ptr . The argument flags is a
combination of bit flags; see Flags for Wordexp, for details of
the flags.
You shouldn't use any of the characters The results of word expansion are a sequence of words. The function
To return this vector, If
|
void wordfree (wordexp_t *word-vector-ptr) | Function |
Free the storage used for the word-strings and vector that
*word-vector-ptr points to. This does not free the
structure *word-vector-ptr itself--only the other
data it points to.
|
This section describes the flags that you can specify in the
flags argument to wordexp
. Choose the flags you want,
and combine them with the C operator |
.
WRDE_APPEND
wordexp
. This way you can effectively expand
several words as if they were concatenated with spaces between them.
In order for appending to work, you must not modify the contents of the
word vector structure between calls to wordexp
. And, if you set
WRDE_DOOFFS
in the first call to wordexp
, you must also
set it when you append to the results.
WRDE_DOOFFS
we_offs
field says how many slots to leave.
The blank slots contain null pointers.
WRDE_NOCMD
WRDE_REUSE
wordexp
.
Instead of allocating a new vector of words, this call to wordexp
will use the vector that already exists (making it larger if necessary).
Note that the vector may move, so it is not safe to save an old pointer
and use it again after calling wordexp
. You must fetch
we_pathv
anew after each call.
WRDE_SHOWERR
wordexp
gives these
commands a standard error stream that discards all output.
WRDE_UNDEF
wordexp
ExampleHere is an example of using wordexp
to expand several strings
and use the results to run a shell command. It also shows the use of
WRDE_APPEND
to concatenate the expansions and of wordfree
to free the space allocated by wordexp
.
int
expand_and_execute (const char *program, const char *options)
{
wordexp_t result;
pid_t pid
int status, i;
/* Expand the string for the program to run. */
switch (wordexp (program, &result, 0))
{
case 0: /* Successful. */
break;
case WRDE_NOSPACE:
/* If the error was WRDE_NOSPACE
,
then perhaps part of the result was allocated. */
wordfree (&result);
default: /* Some other error. */
return -1;
}
/* Expand the strings specified for the arguments. */
for (i = 0; args[i]; i++)
{
if (wordexp (options, &result, WRDE_APPEND))
{
wordfree (&result);
return -1;
}
}
pid = fork ();
if (pid == 0)
{
/* This is the child process. Execute the command. */
execv (result.we_wordv[0], result.we_wordv);
exit (EXIT_FAILURE);
}
else if (pid < 0)
/* The fork failed. Report failure. */
status = -1;
else
/* This is the parent process. Wait for the child to complete. */
if (waitpid (pid, &status, 0) != pid)
status = -1;
wordfree (&result);
return status;
}
It's a standard part of shell syntax that you can use ~
at the
beginning of a file name to stand for your own home directory. You
can use ~user
to stand for user's home directory.
Tilde expansion is the process of converting these abbreviations to the directory names that they stand for.
Tilde expansion applies to the ~
plus all following characters up
to whitespace or a slash. It takes place only at the beginning of a
word, and only if none of the characters to be transformed is quoted in
any way.
Plain ~
uses the value of the environment variable HOME
as the proper home directory name. ~
followed by a user name
uses getpwname
to look up that user in the user database, and
uses whatever directory is recorded there. Thus, ~
followed
by your own name can give different results from plain ~
, if
the value of HOME
is not really your home directory.
Part of ordinary shell syntax is the use of $variable
to
substitute the value of a shell variable into a command. This is called
variable substitution, and it is one part of doing word expansion.
There are two basic ways you can write a variable reference for substitution:
${variable}
${foo}s
expands into tractors
.
$variable
$
. The next punctuation character ends the variable
name. Thus, $foo-bar
refers to the variable foo
and expands
into tractor-bar
.
When you use braces, you can also use various constructs to modify the value that is substituted, or test it in various ways.
${variable:-default}
${variable:=default}
${variable:?message}
Otherwise, print message as an error message on the standard error
stream, and consider word expansion a failure.
${variable:+replacement}
${#variable}
${#foo}
stands for
7
, because tractor
is seven characters.
These variants of variable substitution let you remove part of the variable's value before substituting it. The prefix and suffix are not mere strings; they are wildcard patterns, just like the patterns that you use to match multiple file names. But in this context, they match against parts of the variable value rather than against file names.
${variable%%suffix}
If there is more than one alternative for how to match against suffix, this construct uses the longest possible match.
Thus, ${foo%%r*}
substitutes t
, because the largest
match for r*
at the end of tractor
is ractor
.
${variable%suffix}
If there is more than one alternative for how to match against suffix, this construct uses the shortest possible alternative.
Thus, ${foo%%r*}
substitutes tracto
, because the shortest
match for r*
at the end of tractor
is just r
.
${variable##prefix}
If there is more than one alternative for how to match against prefix, this construct uses the longest possible match.
Thus, ${foo%%r*}
substitutes t
, because the largest
match for r*
at the end of tractor
is ractor
.
${variable#prefix}
If there is more than one alternative for how to match against prefix, this construct uses the shortest possible alternative.
Thus, ${foo%%r*}
substitutes tracto
, because the shortest
match for r*
at the end of tractor
is just r
.
Most programs need to do either input (reading data) or output (writing data), or most frequently both, in order to do anything useful. The GNU C library provides such a large selection of input and output functions that the hardest part is often deciding which function is most appropriate!
This chapter introduces concepts and terminology relating to input and output. Other chapters relating to the GNU I/O facilities are:
Before you can read or write the contents of a file, you must establish a connection or communications channel to the file. This process is called opening the file. You can open a file for reading, writing, or both.
The connection to an open file is represented either as a stream or as a file descriptor. You pass this as an argument to the functions that do the actual read or write operations, to tell them which file to operate on. Certain functions expect streams, and others are designed to operate on file descriptors.
When you have finished reading to or writing from the file, you can terminate the connection by closing the file. Once you have closed a stream or file descriptor, you cannot do any more input or output operations on it.
When you want to do input or output to a file, you have a choice of two
basic mechanisms for representing the connection between your program
and the file: file descriptors and streams. File descriptors are
represented as objects of type int
, while streams are represented
as FILE *
objects.
File descriptors provide a primitive, low-level interface to input and output operations. Both file descriptors and streams can represent a connection to a device (such as a terminal), or a pipe or socket for communicating with another process, as well as a normal file. But, if you want to do control operations that are specific to a particular kind of device, you must use a file descriptor; there are no facilities to use streams in this way. You must also use file descriptors if your program needs to do input or output in special modes, such as nonblocking (or polled) input (see File Status Flags).
Streams provide a higher-level interface, layered on top of the primitive file descriptor facilities. The stream interface treats all kinds of files pretty much alike--the sole exception being the three styles of buffering that you can choose (see Stream Buffering).
The main advantage of using the stream interface is that the set of
functions for performing actual input and output operations (as opposed
to control operations) on streams is much richer and more powerful than
the corresponding facilities for file descriptors. The file descriptor
interface provides only simple functions for transferring blocks of
characters, but the stream interface also provides powerful formatted
input and output functions (printf
and scanf
) as well as
functions for character- and line-oriented input and output.
Since streams are implemented in terms of file descriptors, you can extract the file descriptor from a stream and perform low-level operations directly on the file descriptor. You can also initially open a connection as a file descriptor and then make a stream associated with that file descriptor.
In general, you should stick with using streams rather than file descriptors, unless there is some specific operation you want to do that can only be done on a file descriptor. If you are a beginning programmer and aren't sure what functions to use, we suggest that you concentrate on the formatted input functions (see Formatted Input) and formatted output functions (see Formatted Output).
If you are concerned about portability of your programs to systems other than GNU, you should also be aware that file descriptors are not as portable as streams. You can expect any system running ISO C to support streams, but non-GNU systems may not support file descriptors at all, or may only implement a subset of the GNU functions that operate on file descriptors. Most of the file descriptor functions in the GNU library are included in the POSIX.1 standard, however.
One of the attributes of an open file is its file position that keeps track of where in the file the next character is to be read or written. In the GNU system, and all POSIX.1 systems, the file position is simply an integer representing the number of bytes from the beginning of the file.
The file position is normally set to the beginning of the file when it is opened, and each time a character is read or written, the file position is incremented. In other words, access to the file is normally sequential.
Ordinary files permit read or write operations at any position within
the file. Some other kinds of files may also permit this. Files which
do permit this are sometimes referred to as random-access files.
You can change the file position using the fseek
function on a
stream (see File Positioning) or the lseek
function on a file
descriptor (see I/O Primitives). If you try to change the file
position on a file that doesn't support random access, you get the
ESPIPE
error.
Streams and descriptors that are opened for append access are treated specially for output: output to such files is always appended sequentially to the end of the file, regardless of the file position. However, the file position is still used to control where in the file reading is done.
If you think about it, you'll realize that several programs can read a given file at the same time. In order for each program to be able to read the file at its own pace, each program must have its own file pointer, which is not affected by anything the other programs do.
In fact, each opening of a file creates a separate file position. Thus, if you open a file twice even in the same program, you get two streams or descriptors with independent file positions.
By contrast, if you open a descriptor and then duplicate it to get another descriptor, these two descriptors share the same file position: changing the file position of one descriptor will affect the other.
In order to open a connection to a file, or to perform other operations such as deleting a file, you need some way to refer to the file. Nearly all files have names that are strings--even files which are actually devices such as tape drives or terminals. These strings are called file names. You specify the file name to say which file you want to open or operate on.
This section describes the conventions for file names and how the operating system works with them.
In order to understand the syntax of file names, you need to understand how the file system is organized into a hierarchy of directories.
A directory is a file that contains information to associate other files with names; these associations are called links or directory entries. Sometimes, people speak of "files in a directory", but in reality, a directory only contains pointers to files, not the files themselves.
The name of a file contained in a directory entry is called a file
name component. In general, a file name consists of a sequence of one
or more such components, separated by the slash character (/
). A
file name which is just one component names a file with respect to its
directory. A file name with multiple components names a directory, and
then a file in that directory, and so on.
Some other documents, such as the POSIX standard, use the term
pathname for what we call a file name, and either filename
or pathname component for what this manual calls a file name
component. We don't use this terminology because a "path" is
something completely different (a list of directories to search), and we
think that "pathname" used for something else will confuse users. We
always use "file name" and "file name component" (or sometimes just
"component", where the context is obvious) in GNU documentation. Some
macros use the POSIX terminology in their names, such as
PATH_MAX
. These macros are defined by the POSIX standard, so we
cannot change their names.
You can find more detailed information about operations on directories in File System Interface.
A file name consists of file name components separated by slash
(/
) characters. On the systems that the GNU C library supports,
multiple successive /
characters are equivalent to a single
/
character.
The process of determining what file a file name refers to is called file name resolution. This is performed by examining the components that make up a file name in left-to-right order, and locating each successive component in the directory named by the previous component. Of course, each of the files that are referenced as directories must actually exist, be directories instead of regular files, and have the appropriate permissions to be accessible by the process; otherwise the file name resolution fails.
If a file name begins with a /
, the first component in the file
name is located in the root directory of the process (usually all
processes on the system have the same root directory). Such a file name
is called an absolute file name.
Otherwise, the first component in the file name is located in the current working directory (see Working Directory). This kind of file name is called a relative file name.
The file name components .
("dot") and ..
("dot-dot")
have special meanings. Every directory has entries for these file name
components. The file name component .
refers to the directory
itself, while the file name component ..
refers to its
parent directory (the directory that contains the link for the
directory in question). As a special case, ..
in the root
directory refers to the root directory itself, since it has no parent;
thus /..
is the same as /
.
Here are some examples of file names:
/a
a
, in the root directory.
/a/b
b
, in the directory named a
in the root directory.
a
a
, in the current working directory.
/a/./b
/a/b
.
./a
a
, in the current working directory.
../a
a
, in the parent directory of the current working
directory.
A file name that names a directory may optionally end in a /
.
You can specify a file name of /
to refer to the root directory,
but the empty string is not a meaningful file name. If you want to
refer to the current working directory, use a file name of .
or
./
.
Unlike some other operating systems, the GNU system doesn't have any
built-in support for file types (or extensions) or file versions as part
of its file name syntax. Many programs and utilities use conventions
for file names--for example, files containing C source code usually
have names suffixed with .c
--but there is nothing in the file
system itself that enforces this kind of convention.
Functions that accept file name arguments usually detect these
errno
error conditions relating to the file name syntax or
trouble finding the named file. These errors are referred to throughout
this manual as the usual file name errors.
EACCES
ENAMETOOLONG
PATH_MAX
, or when an individual file name component
has a length greater than NAME_MAX
. See Limits for Files.
In the GNU system, there is no imposed limit on overall file name
length, but some file systems may place limits on the length of a
component.
ENOENT
ENOTDIR
ELOOP
The rules for the syntax of file names discussed in File Names, are the rules normally used by the GNU system and by other POSIX systems. However, other operating systems may use other conventions.
There are two reasons why it can be important for you to be aware of file name portability issues:
The ISO C standard says very little about file name syntax, only that file names are strings. In addition to varying restrictions on the length of file names and what characters can validly appear in a file name, different operating systems use different conventions and syntax for concepts such as structured directories and file types or extensions. Some concepts such as file versions might be supported in some operating systems and not by others.
The POSIX.1 standard allows implementations to put additional restrictions on file name syntax, concerning what characters are permitted in file names and on the length of file name and file name component strings. However, in the GNU system, you do not need to worry about these restrictions; any character except the null character is permitted in a file name string, and there are no limits on the length of file name strings.
This chapter describes the functions for creating streams and performing input and output operations on them. As discussed in I/O Overview, a stream is a fairly abstract, high-level concept representing a communications channel to a file, device, or process.
printf
and related functions.
printf
and friends.
scanf
and related functions.
For historical reasons, the type of the C data structure that represents
a stream is called FILE
rather than "stream". Since most of
the library functions deal with objects of type FILE *
, sometimes
the term file pointer is also used to mean "stream". This leads
to unfortunate confusion over terminology in many books on C. This
manual, however, is careful to use the terms "file" and "stream"
only in the technical sense.
The FILE
type is declared in the header file stdio.h
.
FILE | Data Type |
This is the data type used to represent stream objects. A FILE
object holds all of the internal state information about the connection
to the associated file, including such things as the file position
indicator and buffering information. Each stream also has error and
end-of-file status indicators that can be tested with the ferror
and feof functions; see EOF and Errors.
|
FILE
objects are allocated and managed internally by the
input/output library functions. Don't try to create your own objects of
type FILE
; let the library do it. Your programs should
deal only with pointers to these objects (that is, FILE *
values)
rather than the objects themselves.
When the main
function of your program is invoked, it already has
three predefined streams open and available for use. These represent
the "standard" input and output channels that have been established
for the process.
These streams are declared in the header file stdio.h
.
FILE * stdin | Variable |
The standard input stream, which is the normal source of input for the program. |
FILE * stdout | Variable |
The standard output stream, which is used for normal output from the program. |
FILE * stderr | Variable |
The standard error stream, which is used for error messages and diagnostics issued by the program. |
In the GNU system, you can specify what files or processes correspond to these streams using the pipe and redirection facilities provided by the shell. (The primitives shells use to implement these facilities are described in File System Interface.) Most other operating systems provide similar mechanisms, but the details of how to use them can vary.
In the GNU C library, stdin
, stdout
, and stderr
are
normal variables which you can set just like any others. For example,
to redirect the standard output to a file, you could do:
fclose (stdout); stdout = fopen ("standard-output-file", "w");
Note however, that in other systems stdin
, stdout
, and
stderr
are macros that you cannot assign to in the normal way.
But you can use freopen
to get the effect of closing one and
reopening it. See Opening Streams.
The three streams stdin
, stdout
, and stderr
are not
unoriented at program start (see Streams and I18N).
Opening a file with the fopen
function creates a new stream and
establishes a connection between the stream and a file. This may
involve creating a new file.
Everything described in this section is declared in the header file
stdio.h
.
FILE * fopen (const char *filename, const char *opentype) | Function |
The fopen function opens a stream for I/O to the file
filename, and returns a pointer to the stream.
The opentype argument is a string that controls how the file is opened and specifies attributes of the resulting stream. It must begin with one of the following sequences of characters:
As you can see, Additional characters may appear after these to specify flags for the
call. Always put the mode ( The GNU C library defines one additional character for use in
opentype: the character The character If the opentype string contains the sequence
Any other characters in opentype are simply ignored. They may be meaningful in other systems. If the open fails, When the sources are compiling with |
You can have multiple streams (or file descriptors) pointing to the same file open at the same time. If you do only input, this works straightforwardly, but you must be careful if any output streams are included. See Stream/Descriptor Precautions. This is equally true whether the streams are in one program (not usual) or in several programs (which can easily happen). It may be advantageous to use the file locking facilities to avoid simultaneous access. See File Locks.
FILE * fopen64 (const char *filename, const char *opentype) | Function |
This function is similar to fopen but the stream it returns a
pointer for is opened using open64 . Therefore this stream can be
used even on files larger then 2^31 bytes on 32 bit machines.
Please note that the return type is still If the sources are compiled with |
int FOPEN_MAX | Macro |
The value of this macro is an integer constant expression that
represents the minimum number of streams that the implementation
guarantees can be open simultaneously. You might be able to open more
than this many streams, but that is not guaranteed. The value of this
constant is at least eight, which includes the three standard streams
stdin , stdout , and stderr . In POSIX.1 systems this
value is determined by the OPEN_MAX parameter; see General Limits. In BSD and GNU, it is controlled by the RLIMIT_NOFILE
resource limit; see Limits on Resources.
|
FILE * freopen (const char *filename, const char *opentype, FILE *stream) | Function |
This function is like a combination of fclose and fopen .
It first closes the stream referred to by stream, ignoring any
errors that are detected in the process. (Because errors are ignored,
you should not use freopen on an output stream if you have
actually done any output using the stream.) Then the file named by
filename is opened with mode opentype as for fopen ,
and associated with the same stream object stream.
If the operation fails, a null pointer is returned; otherwise,
When the sources are compiling with |
FILE * freopen64 (const char *filename, const char *opentype, FILE *stream) | Function |
This function is similar to freopen . The only difference is that
on 32 bit machine the stream returned is able to read beyond the
2^31 bytes limits imposed by the normal interface. It should be
noted that the stream pointed to by stream need not be opened
using fopen64 or freopen64 since its mode is not important
for this function.
If the sources are compiled with |
In some situations it is useful to know whether a given stream is available for reading or writing. This information is normally not available and would have to be remembered separately. Solaris introduced a few functions to get this information from the stream descriptor and these functions are also available in the GNU C library.
int __freadable (FILE *stream) | Function |
The __freadable function determines whether the stream
stream was opened to allow reading. In this case the return value
is nonzero. For write-only streams the function returns zero.
This function is declared in |
int __fwritable (FILE *stream) | Function |
The __fwritable function determines whether the stream
stream was opened to allow writing. In this case the return value
is nonzero. For read-only streams the function returns zero.
This function is declared in |
For slightly different kind of problems there are two more functions. They provide even finer-grained information.
int __freading (FILE *stream) | Function |
The __freading function determines whether the stream
stream was last read from or whether it is opened read-only. In
this case the return value is nonzero, otherwise it is zero.
Determining whether a stream opened for reading and writing was last
used for writing allows to draw conclusions about the content about the
buffer, among other things.
This function is declared in |
int __fwriting (FILE *stream) | Function |
The __fwriting function determines whether the stream
stream was last written to or whether it is opened write-only. In
this case the return value is nonzero, otherwise it is zero.
This function is declared in |
When a stream is closed with fclose
, the connection between the
stream and the file is canceled. After you have closed a stream, you
cannot perform any additional operations on it.
int fclose (FILE *stream) | Function |
This function causes stream to be closed and the connection to
the corresponding file to be broken. Any buffered output is written
and any buffered input is discarded. The fclose function returns
a value of 0 if the file was closed successfully, and EOF
if an error was detected.
It is important to check for errors when you call The function |
To close all streams currently available the GNU C Library provides another function.
int fcloseall (void) | Function |
This function causes all open streams of the process to be closed and
the connection to corresponding files to be broken. All buffered data
is written and any buffered input is discarded. The fcloseall
function returns a value of 0 if all the files were closed
successfully, and EOF if an error was detected.
This function should be used only in special situations, e.g., when an error occurred and the program must be aborted. Normally each single stream should be closed separately so that problems with individual streams can be identified. It is also problematic since the standard streams (see Standard Streams) will also be closed. The function |
If the main
function to your program returns, or if you call the
exit
function (see Normal Termination), all open streams are
automatically closed properly. If your program terminates in any other
manner, such as by calling the abort
function (see Aborting a Program) or from a fatal signal (see Signal Handling), open streams
might not be closed properly. Buffered output might not be flushed and
files may be incomplete. For more information on buffering of streams,
see Stream Buffering.
Streams can be used in multi-threaded applications in the same way they are used in single-threaded applications. But the programmer must be aware of a the possible complications. It is important to know about these also if the program one writes never use threads since the design and implementation of many stream functions is heavily influenced by the requirements added by multi-threaded programming.
The POSIX standard requires that by default the stream operations are atomic. I.e., issuing two stream operations for the same stream in two threads at the same time will cause the operations to be executed as if they were issued sequentially. The buffer operations performed while reading or writing are protected from other uses of the same stream. To do this each stream has an internal lock object which has to be (implicitly) acquired before any work can be done.
But there are situations where this is not enough and there are also situations where this is not wanted. The implicit locking is not enough if the program requires more than one stream function call to happen atomically. One example would be if an output line a program wants to generate is created by several function calls. The functions by themselves would ensure only atomicity of their own operation, but not atomicity over all the function calls. For this it is necessary to perform the stream locking in the application code.
void flockfile (FILE *stream) | Function |
The flockfile function acquires the internal locking object
associated with the stream stream. This ensures that no other
thread can explicitly through flockfile /ftrylockfile or
implicit through a call of a stream function lock the stream. The
thread will block until the lock is acquired. An explicit call to
funlockfile has to be used to release the lock.
|
int ftrylockfile (FILE *stream) | Function |
The ftrylockfile function tries to acquire the internal locking
object associated with the stream stream just like
flockfile . But unlike flockfile this function does not
block if the lock is not available. ftrylockfile returns zero if
the lock was successfully acquired. Otherwise the stream is locked by
another thread.
|
void funlockfile (FILE *stream) | Function |
The funlockfile function releases the internal locking object of
the stream stream. The stream must have been locked before by a
call to flockfile or a successful call of ftrylockfile .
The implicit locking performed by the stream operations do not count.
The funlockfile function does not return an error status and the
behavior of a call for a stream which is not locked by the current
thread is undefined.
|
The following example shows how the functions above can be used to
generate an output line atomically even in multi-threaded applications
(yes, the same job could be done with one fprintf
call but it is
sometimes not possible):
FILE *fp; { ... flockfile (fp); fputs ("This is test number ", fp); fprintf (fp, "%d\n", test); funlockfile (fp) }
Without the explicit locking it would be possible for another thread to
use the stream fp after the fputs
call return and before
fprintf
was called with the result that the number does not
follow the word number
.
From this description it might already be clear that the locking objects
in streams are no simple mutexes. Since locking the same stream twice
in the same thread is allowed the locking objects must be equivalent to
recursive mutexes. These mutexes keep track of the owner and the number
of times the lock is acquired. The same number of funlockfile
calls by the same threads is necessary to unlock the stream completely.
For instance:
void foo (FILE *fp) { ftrylockfile (fp); fputs ("in foo\n", fp); /* This is very wrong!!! */ funlockfile (fp); }
It is important here that the funlockfile
function is only called
if the ftrylockfile
function succeeded in locking the stream. It
is therefore always wrong to ignore the result of ftrylockfile
.
And it makes no sense since otherwise one would use flockfile
.
The result of code like that above is that either funlockfile
tries to free a stream that hasn't been locked by the current thread or it
frees the stream prematurely. The code should look like this:
void foo (FILE *fp) { if (ftrylockfile (fp) == 0) { fputs ("in foo\n", fp); funlockfile (fp); } }
Now that we covered why it is necessary to have these locking it is necessary to talk about situations when locking is unwanted and what can be done. The locking operations (explicit or implicit) don't come for free. Even if a lock is not taken the cost is not zero. The operations which have to be performed require memory operations that are safe in multi-processor environments. With the many local caches involved in such systems this is quite costly. So it is best to avoid the locking completely if it is not needed - because the code in question is never used in a context where two or more threads may use a stream at a time. This can be determined most of the time for application code; for library code which can be used in many contexts one should default to be conservative and use locking.
There are two basic mechanisms to avoid locking. The first is to use
the _unlocked
variants of the stream operations. The POSIX
standard defines quite a few of those and the GNU library adds a few
more. These variants of the functions behave just like the functions
with the name without the suffix except that they do not lock the
stream. Using these functions is very desirable since they are
potentially much faster. This is not only because the locking
operation itself is avoided. More importantly, functions like
putc
and getc
are very simple and traditionally (before the
introduction of threads) were implemented as macros which are very fast
if the buffer is not empty. With the addition of locking requirements
these functions are no longer implemented as macros since they would
would expand to too much code.
But these macros are still available with the same functionality under the new
names putc_unlocked
and getc_unlocked
. This possibly huge
difference of speed also suggests the use of the _unlocked
functions even if locking is required. The difference is that the
locking then has to be performed in the program:
void foo (FILE *fp, char *buf) { flockfile (fp); while (*buf != '/') putc_unlocked (*buf++, fp); funlockfile (fp); }
If in this example the putc
function would be used and the
explicit locking would be missing the putc
function would have to
acquire the lock in every call, potentially many times depending on when
the loop terminates. Writing it the way illustrated above allows the
putc_unlocked
macro to be used which means no locking and direct
manipulation of the buffer of the stream.
A second way to avoid locking is by using a non-standard function which was introduced in Solaris and is available in the GNU C library as well.
int __fsetlocking (FILE *stream, int type) | Function |
The
The return value of This function and the values for the type parameter are declared
in |
This function is especially useful when program code has to be used
which is written without knowledge about the _unlocked
functions
(or if the programmer was too lazy to use them).
ISO C90 introduced the new type wchar_t
to allow handling
larger character sets. What was missing was a possibility to output
strings of wchar_t
directly. One had to convert them into
multibyte strings using mbstowcs
(there was no mbsrtowcs
yet) and then use the normal stream functions. While this is doable it
is very cumbersome since performing the conversions is not trivial and
greatly increases program complexity and size.
The Unix standard early on (I think in XPG4.2) introduced two additional
format specifiers for the printf
and scanf
families of
functions. Printing and reading of single wide characters was made
possible using the %C
specifier and wide character strings can be
handled with %S
. These modifiers behave just like %c
and
%s
only that they expect the corresponding argument to have the
wide character type and that the wide character and string are
transformed into/from multibyte strings before being used.
This was a beginning but it is still not good enough. Not always is it
desirable to use printf
and scanf
. The other, smaller and
faster functions cannot handle wide characters. Second, it is not
possible to have a format string for printf
and scanf
consisting of wide characters. The result is that format strings would
have to be generated if they have to contain non-basic characters.
In the Amendment 1 to ISO C90 a whole new set of functions was
added to solve the problem. Most of the stream functions got a
counterpart which take a wide character or wide character string instead
of a character or string respectively. The new functions operate on the
same streams (like stdout
). This is different from the model of
the C++ runtime library where separate streams for wide and normal I/O
are used.
Being able to use the same stream for wide and normal operations comes
with a restriction: a stream can be used either for wide operations or
for normal operations. Once it is decided there is no way back. Only a
call to freopen
or freopen64
can reset the
orientation. The orientation can be decided in three ways:
fread
and fwrite
functions) the stream is marked as not
wide oriented.
fwide
function can be used to set the orientation either way.
It is important to never mix the use of wide and not wide operations on
a stream. There are no diagnostics issued. The application behavior
will simply be strange or the application will simply crash. The
fwide
function can help avoiding this.
int fwide (FILE *stream, int mode) | Function |
The If mode is zero the current orientation state is queried and nothing is changed. The This function was introduced in Amendment 1 to ISO C90 and is
declared in |
It is generally a good idea to orient a stream as early as possible.
This can prevent surprise especially for the standard streams
stdin
, stdout
, and stderr
. If some library
function in some situations uses one of these streams and this use
orients the stream in a different way the rest of the application
expects it one might end up with hard to reproduce errors. Remember
that no errors are signal if the streams are used incorrectly. Leaving
a stream unoriented after creation is normally only necessary for
library functions which create streams which can be used in different
contexts.
When writing code which uses streams and which can be used in different
contexts it is important to query the orientation of the stream before
using it (unless the rules of the library interface demand a specific
orientation). The following little, silly function illustrates this.
void print_f (FILE *fp) { if (fwide (fp, 0) > 0) /* Positive return value means wide orientation. */ fputwc (L'f', fp); else fputc ('f', fp); }
Note that in this case the function print_f
decides about the
orientation of the stream if it was unoriented before (will not happen
if the advise above is followed).
The encoding used for the wchar_t
values is unspecified and the
user must not make any assumptions about it. For I/O of wchar_t
values this means that it is impossible to write these values directly
to the stream. This is not what follows from the ISO C locale model
either. What happens instead is that the bytes read from or written to
the underlying media are first converted into the internal encoding
chosen by the implementation for wchar_t
. The external encoding
is determined by the LC_CTYPE
category of the current locale or
by the ccs
part of the mode specification given to fopen
,
fopen64
, freopen
, or freopen64
. How and when the
conversion happens is unspecified and it happens invisible to the user.
Since a stream is created in the unoriented state it has at that point
no conversion associated with it. The conversion which will be used is
determined by the LC_CTYPE
category selected at the time the
stream is oriented. If the locales are changed at the runtime this
might produce surprising results unless one pays attention. This is
just another good reason to orient the stream explicitly as soon as
possible, perhaps with a call to fwide
.
This section describes functions for performing character- and line-oriented output.
These narrow streams functions are declared in the header file
stdio.h
and the wide stream functions in wchar.h
.
int fputc (int c, FILE *stream) | Function |
The fputc function converts the character c to type
unsigned char , and writes it to the stream stream.
EOF is returned if a write error occurs; otherwise the
character c is returned.
|
wint_t fputwc (wchar_t wc, FILE *stream) | Function |
The fputwc function writes the wide character wc to the
stream stream. WEOF is returned if a write error occurs;
otherwise the character wc is returned.
|
int fputc_unlocked (int c, FILE *stream) | Function |
The fputc_unlocked function is equivalent to the fputc
function except that it does not implicitly lock the stream.
|
wint_t fputwc_unlocked (wint_t wc, FILE *stream) | Function |
The fputwc_unlocked function is equivalent to the fputwc
function except that it does not implicitly lock the stream.
This function is a GNU extension. |
int putc (int c, FILE *stream) | Function |
This is just like fputc , except that most systems implement it as
a macro, making it faster. One consequence is that it may evaluate the
stream argument more than once, which is an exception to the
general rule for macros. putc is usually the best function to
use for writing a single character.
|
wint_t putwc (wchar_t wc, FILE *stream) | Function |
This is just like fputwc , except that it can be implement as
a macro, making it faster. One consequence is that it may evaluate the
stream argument more than once, which is an exception to the
general rule for macros. putwc is usually the best function to
use for writing a single wide character.
|
int putc_unlocked (int c, FILE *stream) | Function |
The putc_unlocked function is equivalent to the putc
function except that it does not implicitly lock the stream.
|
wint_t putwc_unlocked (wchar_t wc, FILE *stream) | Function |
The putwc_unlocked function is equivalent to the putwc
function except that it does not implicitly lock the stream.
This function is a GNU extension. |
int putchar (int c) | Function |
The putchar function is equivalent to putc with
stdout as the value of the stream argument.
|
wint_t putwchar (wchar_t wc) | Function |
The putwchar function is equivalent to putwc with
stdout as the value of the stream argument.
|
int putchar_unlocked (int c) | Function |
The putchar_unlocked function is equivalent to the putchar
function except that it does not implicitly lock the stream.
|
wint_t putwchar_unlocked (wchar_t wc) | Function |
The putwchar_unlocked function is equivalent to the putwchar
function except that it does not implicitly lock the stream.
This function is a GNU extension. |
int fputs (const char *s, FILE *stream) | Function |
The function fputs writes the string s to the stream
stream. The terminating null character is not written.
This function does not add a newline character, either.
It outputs only the characters in the string.
This function returns For example:
fputs ("Are ", stdout); fputs ("you ", stdout); fputs ("hungry?\n", stdout); outputs the text |
int fputws (const wchar_t *ws, FILE *stream) | Function |
The function fputws writes the wide character string ws to
the stream stream. The terminating null character is not written.
This function does not add a newline character, either. It
outputs only the characters in the string.
This function returns |
int fputs_unlocked (const char *s, FILE *stream) | Function |
The fputs_unlocked function is equivalent to the fputs
function except that it does not implicitly lock the stream.
This function is a GNU extension. |
int fputws_unlocked (const wchar_t *ws, FILE *stream) | Function |
The fputws_unlocked function is equivalent to the fputws
function except that it does not implicitly lock the stream.
This function is a GNU extension. |
int puts (const char *s) | Function |
The puts function writes the string s to the stream
stdout followed by a newline. The terminating null character of
the string is not written. (Note that fputs does not
write a newline as this function does.)
puts ("This is a message."); outputs the text |
int putw (int w, FILE *stream) | Function |
This function writes the word w (that is, an int ) to
stream. It is provided for compatibility with SVID, but we
recommend you use fwrite instead (see Block Input/Output).
|
This section describes functions for performing character-oriented
input. These narrow streams functions are declared in the header file
stdio.h
and the wide character functions are declared in
wchar.h
.
These functions return an int
or wint_t
value (for narrow
and wide stream functions respectively) that is either a character of
input, or the special value EOF
/WEOF
(usually -1). For
the narrow stream functions it is important to store the result of these
functions in a variable of type int
instead of char
, even
when you plan to use it only as a character. Storing EOF
in a
char
variable truncates its value to the size of a character, so
that it is no longer distinguishable from the valid character
(char) -1
. So always use an int
for the result of
getc
and friends, and check for EOF
after the call; once
you've verified that the result is not EOF
, you can be sure that
it will fit in a char
variable without loss of information.
int fgetc (FILE *stream) | Function |
This function reads the next character as an unsigned char from
the stream stream and returns its value, converted to an
int . If an end-of-file condition or read error occurs,
EOF is returned instead.
|
wint_t fgetwc (FILE *stream) | Function |
This function reads the next wide character from the stream stream
and returns its value. If an end-of-file condition or read error
occurs, WEOF is returned instead.
|
int fgetc_unlocked (FILE *stream) | Function |
The fgetc_unlocked function is equivalent to the fgetc
function except that it does not implicitly lock the stream.
|
wint_t fgetwc_unlocked (FILE *stream) | Function |
The fgetwc_unlocked function is equivalent to the fgetwc
function except that it does not implicitly lock the stream.
This function is a GNU extension. |
int getc (FILE *stream) | Function |
This is just like fgetc , except that it is permissible (and
typical) for it to be implemented as a macro that evaluates the
stream argument more than once. getc is often highly
optimized, so it is usually the best function to use to read a single
character.
|
wint_t getwc (FILE *stream) | Function |
This is just like fgetwc , except that it is permissible for it to
be implemented as a macro that evaluates the stream argument more
than once. getwc can be highly optimized, so it is usually the
best function to use to read a single wide character.
|
int getc_unlocked (FILE *stream) | Function |
The getc_unlocked function is equivalent to the getc
function except that it does not implicitly lock the stream.
|
wint_t getwc_unlocked (FILE *stream) | Function |
The getwc_unlocked function is equivalent to the getwc
function except that it does not implicitly lock the stream.
This function is a GNU extension. |
int getchar (void) | Function |
The getchar function is equivalent to getc with stdin
as the value of the stream argument.
|
wint_t getwchar (void) | Function |
The getwchar function is equivalent to getwc with stdin
as the value of the stream argument.
|
int getchar_unlocked (void) | Function |
The getchar_unlocked function is equivalent to the getchar
function except that it does not implicitly lock the stream.
|
wint_t getwchar_unlocked (void) | Function |
The getwchar_unlocked function is equivalent to the getwchar
function except that it does not implicitly lock the stream.
This function is a GNU extension. |
Here is an example of a function that does input using fgetc
. It
would work just as well using getc
instead, or using
getchar ()
instead of fgetc (stdin)
. The code would
also work the same for the wide character stream functions.
int y_or_n_p (const char *question) { fputs (question, stdout); while (1) { int c, answer; /* Write a space to separate answer from question. */ fputc (' ', stdout); /* Read the first character of the line. This should be the answer character, but might not be. */ c = tolower (fgetc (stdin)); answer = c; /* Discard rest of input line. */ while (c != '\n' && c != EOF) c = fgetc (stdin); /* Obey the answer if it was valid. */ if (answer == 'y') return 1; if (answer == 'n') return 0; /* Answer was invalid: ask for valid answer. */ fputs ("Please answer y or n:", stdout); } }
int getw (FILE *stream) | Function |
This function reads a word (that is, an int ) from stream.
It's provided for compatibility with SVID. We recommend you use
fread instead (see Block Input/Output). Unlike getc ,
any int value could be a valid result. getw returns
EOF when it encounters end-of-file or an error, but there is no
way to distinguish this from an input word with value -1.
|
Since many programs interpret input on the basis of lines, it is convenient to have functions to read a line of text from a stream.
Standard C has functions to do this, but they aren't very safe: null
characters and even (for gets
) long lines can confuse them. So
the GNU library provides the nonstandard getline
function that
makes it easy to read lines reliably.
Another GNU extension, getdelim
, generalizes getline
. It
reads a delimited record, defined as everything through the next
occurrence of a specified delimiter character.
All these functions are declared in stdio.h
.
ssize_t getline (char **lineptr, size_t *n, FILE *stream) | Function |
This function reads an entire line from stream, storing the text
(including the newline and a terminating null character) in a buffer
and storing the buffer address in *lineptr .
Before calling If you set In either case, when When This function is a GNU extension, but it is the recommended way to read lines from a stream. The alternative standard functions are unreliable. If an error occurs or end of file is reached without any bytes read,
|
ssize_t getdelim (char **lineptr, size_t *n, int delimiter, FILE *stream) | Function |
This function is like getline except that the character which
tells it to stop reading is not necessarily newline. The argument
delimiter specifies the delimiter character; getdelim keeps
reading until it sees that character (or end of file).
The text is stored in lineptr, including the delimiter character
and a terminating null. Like
ssize_t getline (char **lineptr, size_t *n, FILE *stream) { return getdelim (lineptr, n, '\n', stream); } |
char * fgets (char *s, int count, FILE *stream) | Function |
The fgets function reads characters from the stream stream
up to and including a newline character and stores them in the string
s, adding a null character to mark the end of the string. You
must supply count characters worth of space in s, but the
number of characters read is at most count - 1. The extra
character space is used to hold the null character at the end of the
string.
If the system is already at end of file when you call Warning: If the input data has a null character, you can't tell.
So don't use |
wchar_t * fgetws (wchar_t *ws, int count, FILE *stream) | Function |
The fgetws function reads wide characters from the stream
stream up to and including a newline character and stores them in
the string ws, adding a null wide character to mark the end of the
string. You must supply count wide characters worth of space in
ws, but the number of characters read is at most count
- 1. The extra character space is used to hold the null wide
character at the end of the string.
If the system is already at end of file when you call Warning: If the input data has a null wide character (which are
null bytes in the input stream), you can't tell. So don't use
|
char * fgets_unlocked (char *s, int count, FILE *stream) | Function |
The fgets_unlocked function is equivalent to the fgets
function except that it does not implicitly lock the stream.
This function is a GNU extension. |
wchar_t * fgetws_unlocked (wchar_t *ws, int count, FILE *stream) | Function |
The fgetws_unlocked function is equivalent to the fgetws
function except that it does not implicitly lock the stream.
This function is a GNU extension. |
char * gets (char *s) | Deprecated function |
The function gets reads characters from the stream stdin
up to the next newline character, and stores them in the string s.
The newline character is discarded (note that this differs from the
behavior of fgets , which copies the newline character into the
string). If gets encounters a read error or end-of-file, it
returns a null pointer; otherwise it returns s.
Warning: The |
In parser programs it is often useful to examine the next character in the input stream without removing it from the stream. This is called "peeking ahead" at the input because your program gets a glimpse of the input it will read next.
Using stream I/O, you can peek ahead at input by first reading it and
then unreading it (also called pushing it back on the stream).
Unreading a character makes it available to be input again from the stream,
by the next call to fgetc
or other input function on that stream.
ungetc
to do unreading.
Here is a pictorial explanation of unreading. Suppose you have a
stream reading a file that contains just six characters, the letters
foobar
. Suppose you have read three characters so far. The
situation looks like this:
f o o b a r ^
so the next input character will be b
.
If instead of reading b
you unread the letter o
, you get a
situation like this:
f o o b a r | o-- ^
so that the next input characters will be o
and b
.
If you unread 9
instead of o
, you get this situation:
f o o b a r | 9-- ^
so that the next input characters will be 9
and b
.
ungetc
To Do UnreadingThe function to unread a character is called ungetc
, because it
reverses the action of getc
.
int ungetc (int c, FILE *stream) | Function |
The ungetc function pushes back the character c onto the
input stream stream. So the next input from stream will
read c before anything else.
If c is The character that you push back doesn't have to be the same as the last
character that was actually read from the stream. In fact, it isn't
necessary to actually read any characters from the stream before
unreading them with The GNU C library only supports one character of pushback--in other
words, it does not work to call Pushing back characters doesn't alter the file; only the internal
buffering for the stream is affected. If a file positioning function
(such as Unreading a character on a stream that is at end of file clears the end-of-file indicator for the stream, because it makes the character of input available. After you read that character, trying to read again will encounter end of file. |
wint_t ungetwc (wint_t wc, FILE *stream) | Function |
The ungetwc function behaves just like ungetc just that it
pushes back a wide character.
|
Here is an example showing the use of getc
and ungetc
to
skip over whitespace characters. When this function reaches a
non-whitespace character, it unreads that character to be seen again on
the next read operation on the stream.
#include <stdio.h> #include <ctype.h> void skip_whitespace (FILE *stream) { int c; do /* No need to check forEOF
because it is notisspace
, andungetc
ignoresEOF
. */ c = getc (stream); while (isspace (c)); ungetc (c, stream); }
This section describes how to do input and output operations on blocks of data. You can use these functions to read and write binary data, as well as to read and write text in fixed-size blocks instead of by characters or lines.
Binary files are typically used to read and write blocks of data in the same format as is used to represent the data in a running program. In other words, arbitrary blocks of memory--not just character or string objects--can be written to a binary file, and meaningfully read in again by the same program.
Storing data in binary form is often considerably more efficient than using the formatted I/O functions. Also, for floating-point numbers, the binary form avoids possible loss of precision in the conversion process. On the other hand, binary files can't be examined or modified easily using many standard file utilities (such as text editors), and are not portable between different implementations of the language, or different kinds of computers.
These functions are declared in stdio.h
.
size_t fread (void *data, size_t size, size_t count, FILE *stream) | Function |
This function reads up to count objects of size size into
the array data, from the stream stream. It returns the
number of objects actually read, which might be less than count if
a read error occurs or the end of the file is reached. This function
returns a value of zero (and doesn't read anything) if either size
or count is zero.
If |
size_t fread_unlocked (void *data, size_t size, size_t count, FILE *stream) | Function |
The fread_unlocked function is equivalent to the fread
function except that it does not implicitly lock the stream.
This function is a GNU extension. |
size_t fwrite (const void *data, size_t size, size_t count, FILE *stream) | Function |
This function writes up to count objects of size size from the array data, to the stream stream. The return value is normally count, if the call succeeds. Any other value indicates some sort of error, such as running out of space. |
size_t fwrite_unlocked (const void *data, size_t size, size_t count, FILE *stream) | Function |
The fwrite_unlocked function is equivalent to the fwrite
function except that it does not implicitly lock the stream.
This function is a GNU extension. |
The functions described in this section (printf
and related
functions) provide a convenient way to perform formatted output. You
call printf
with a format string or template string
that specifies how to format the values of the remaining arguments.
Unless your program is a filter that specifically performs line- or
character-oriented processing, using printf
or one of the other
related functions described in this section is usually the easiest and
most concise way to perform output. These functions are especially
useful for printing error messages, tables of data, and the like.
vprintf
and friends.
parse_printf_format
.
The printf
function can be used to print any number of arguments.
The template string argument you supply in a call provides
information not only about the number of additional arguments, but also
about their types and what style should be used for printing them.
Ordinary characters in the template string are simply written to the
output stream as-is, while conversion specifications introduced by
a %
character in the template cause subsequent arguments to be
formatted and written to the output stream. For example,
int pct = 37; char filename[] = "foo.txt"; printf ("Processing of `%s' is %d%% finished.\nPlease be patient.\n", filename, pct);
produces output like
Processing of `foo.txt' is 37% finished. Please be patient.
This example shows the use of the %d
conversion to specify that
an int
argument should be printed in decimal notation, the
%s
conversion to specify printing of a string argument, and
the %%
conversion to print a literal %
character.
There are also conversions for printing an integer argument as an
unsigned value in octal, decimal, or hexadecimal radix (%o
,
%u
, or %x
, respectively); or as a character value
(%c
).
Floating-point numbers can be printed in normal, fixed-point notation
using the %f
conversion or in exponential notation using the
%e
conversion. The %g
conversion uses either %e
or %f
format, depending on what is more appropriate for the
magnitude of the particular number.
You can control formatting more precisely by writing modifiers
between the %
and the character that indicates which conversion
to apply. These slightly alter the ordinary behavior of the conversion.
For example, most conversion specifications permit you to specify a
minimum field width and a flag indicating whether you want the result
left- or right-justified within the field.
The specific flags and modifiers that are permitted and their interpretation vary depending on the particular conversion. They're all described in more detail in the following sections. Don't worry if this all seems excessively complicated at first; you can almost always get reasonable free-format output without using any of the modifiers at all. The modifiers are mostly used to make the output look "prettier" in tables.
This section provides details about the precise syntax of conversion
specifications that can appear in a printf
template
string.
Characters in the template string that are not part of a conversion specification are printed as-is to the output stream. Multibyte character sequences (see Character Set Handling) are permitted in a template string.
The conversion specifications in a printf
template string have
the general form:
% [ param-no $] flags width [ . precision ] type conversion
For example, in the conversion specifier %-10.8ld
, the -
is a flag, 10
specifies the field width, the precision is
8
, the letter l
is a type modifier, and d
specifies
the conversion style. (This particular type specifier says to
print a long int
argument in decimal notation, with a minimum of
8 digits left-justified in a field at least 10 characters wide.)
In more detail, output conversion specifications consist of an
initial %
character followed in sequence by:
printf
function are assigned to the
formats in the order of appearance in the format string. But in some
situations (such as message translation) this is not desirable and this
extension allows an explicit parameter to be specified.
The param-no part of the format must be an integer in the range of 1 to the maximum number of arguments present to the function call. Some implementations limit this number to a certainly upper bound. The exact limit can be retrieved by the following constant.
NL_ARGMAX | Macro |
The value of ARGMAX is the maximum value allowed for the
specification of an positional parameter in a printf call. The
actual value in effect at runtime can be retrieved by using
sysconf using the _SC_NL_ARGMAX parameter see Sysconf Definition.
Some system have a quite low limit such as 9 for System V systems. The GNU C library has no real limit. |
If any of the formats has a specification for the parameter position all of them in the format string shall have one. Otherwise the behavior is undefined.
You can also specify a field width of *
. This means that the
next argument in the argument list (before the actual value to be
printed) is used as the field width. The value must be an int
.
If the value is negative, this means to set the -
flag (see
below) and to use the absolute value as the field width.
.
) followed optionally by a decimal integer
(which defaults to zero if omitted).
You can also specify a precision of *
. This means that the next
argument in the argument list (before the actual value to be printed) is
used as the precision. The value must be an int
, and is ignored
if it is negative. If you specify *
for both the field width and
precision, the field width argument precedes the precision argument.
Other C library versions may not recognize this syntax.
int
,
but you can specify h
, l
, or L
for other integer
types.)
The exact options that are permitted and how they are interpreted vary between the different conversion specifiers. See the descriptions of the individual conversions for information about the particular options that they use.
With the -Wformat
option, the GNU C compiler checks calls to
printf
and related functions. It examines the format string and
verifies that the correct number and types of arguments are supplied.
There is also a GNU C syntax to tell the compiler that a function you
write uses a printf
-style format string.
See Declaring Attributes of Functions, for more information.
Here is a table summarizing what all the different conversions do:
%d
, %i
%d
and %i
are synonymous for
output, but are different when used with scanf
for input
(see Table of Input Conversions).
%o
%u
%x
, %X
%x
uses
lower-case letters and %X
uses upper-case. See Integer Conversions, for details.
%f
%e
, %E
%e
uses
lower-case letters and %E
uses upper-case. See Floating-Point Conversions, for details.
%g
, %G
%g
uses
lower-case letters and %G
uses upper-case. See Floating-Point Conversions, for details.
%a
, %A
%a
uses
lower-case letters and %A
uses upper-case. See Floating-Point Conversions, for details.
%c
%C
%lc
which is supported for compatibility
with the Unix standard.
%s
%S
%ls
which is supported for compatibility
with the Unix standard.
%p
%n
%m
errno
.
(This is a GNU extension.)
See Other Output Conversions.
%%
%
character. See Other Output Conversions.
If the syntax of a conversion specification is invalid, unpredictable things will happen, so don't do this. If there aren't enough function arguments provided to supply values for all the conversion specifications in the template string, or if the arguments are not of the correct types, the results are unpredictable. If you supply more arguments than conversion specifications, the extra argument values are simply ignored; this is sometimes useful.
This section describes the options for the %d
, %i
,
%o
, %u
, %x
, and %X
conversion
specifications. These conversions print integers in various formats.
The %d
and %i
conversion specifications both print an
int
argument as a signed decimal number; while %o
,
%u
, and %x
print the argument as an unsigned octal,
decimal, or hexadecimal number (respectively). The %X
conversion
specification is just like %x
except that it uses the characters
ABCDEF
as digits instead of abcdef
.
The following flags are meaningful:
-
+
%d
and %i
conversions, print a
plus sign if the value is positive.
%d
and %i
conversions, if the result
doesn't start with a plus or minus sign, prefix it with a space
character instead. Since the +
flag ensures that the result
includes a sign, this flag is ignored if you supply both of them.
#
%o
conversion, this forces the leading digit to be
0
, as if by increasing the precision. For %x
or
%X
, this prefixes a leading 0x
or 0X
(respectively)
to the result. This doesn't do anything useful for the %d
,
%i
, or %u
conversions. Using this flag produces output
which can be parsed by the strtoul
function (see Parsing of Integers) and scanf
with the %i
conversion
(see Numeric Input Conversions).
'
LC_NUMERIC
category; see General Numeric. This flag is a
GNU extension.
0
-
flag is also specified, or if a precision is specified.
If a precision is supplied, it specifies the minimum number of digits to appear; leading zeros are produced if necessary. If you don't specify a precision, the number is printed with as many digits as it needs. If you convert a value of zero with an explicit precision of zero, then no characters at all are produced.
Without a type modifier, the corresponding argument is treated as an
int
(for the signed conversions %i
and %d
) or
unsigned int
(for the unsigned conversions %o
, %u
,
%x
, and %X
). Recall that since printf
and friends
are variadic, any char
and short
arguments are
automatically converted to int
by the default argument
promotions. For arguments of other integer types, you can use these
modifiers:
hh
signed char
or unsigned
char
, as appropriate. A char
argument is converted to an
int
or unsigned int
by the default argument promotions
anyway, but the h
modifier says to convert it back to a
char
again.
This modifier was introduced in ISO C99.
h
short int
or unsigned
short int
, as appropriate. A short
argument is converted to an
int
or unsigned int
by the default argument promotions
anyway, but the h
modifier says to convert it back to a
short
again.
j
intmax_t
or uintmax_t
, as
appropriate.
This modifier was introduced in ISO C99.
l
long int
or unsigned long
int
, as appropriate. Two l
characters is like the L
modifier, below.
If used with %c
or %s
the corresponding parameter is
considered as a wide character or wide character string respectively.
This use of l
was introduced in Amendment 1 to ISO C90.
L
ll
q
long long int
. (This type is
an extension supported by the GNU C compiler. On systems that don't
support extra-long integers, this is the same as long int
.)
The q
modifier is another name for the same thing, which comes
from 4.4 BSD; a long long int
is sometimes called a "quad"
int
.
t
ptrdiff_t
.
This modifier was introduced in ISO C99.
z
Z
size_t
.
z
was introduced in ISO C99. Z
is a GNU extension
predating this addition and should not be used in new code.
Here is an example. Using the template string:
"|%5d|%-5d|%+5d|%+-5d|% 5d|%05d|%5.0d|%5.2d|%d|\n"
to print numbers using the different options for the %d
conversion gives results like:
| 0|0 | +0|+0 | 0|00000| | 00|0| | 1|1 | +1|+1 | 1|00001| 1| 01|1| | -1|-1 | -1|-1 | -1|-0001| -1| -01|-1| |100000|100000|+100000|+100000| 100000|100000|100000|100000|100000|
In particular, notice what happens in the last case where the number is too large to fit in the minimum field width specified.
Here are some more examples showing how unsigned integers print under
various format options, using the template string:
"|%5u|%5o|%5x|%5X|%#5o|%#5x|%#5X|%#10.8x|\n"
| 0| 0| 0| 0| 0| 0| 0| 00000000| | 1| 1| 1| 1| 01| 0x1| 0X1|0x00000001| |100000|303240|186a0|186A0|0303240|0x186a0|0X186A0|0x000186a0|
This section discusses the conversion specifications for floating-point
numbers: the %f
, %e
, %E
, %g
, and %G
conversions.
The %f
conversion prints its argument in fixed-point notation,
producing output of the form
[-
]ddd.
ddd,
where the number of digits following the decimal point is controlled
by the precision you specify.
The %e
conversion prints its argument in exponential notation,
producing output of the form
[-
]d.
ddde
[+
|-
]dd.
Again, the number of digits following the decimal point is controlled by
the precision. The exponent always contains at least two digits. The
%E
conversion is similar but the exponent is marked with the letter
E
instead of e
.
The %g
and %G
conversions print the argument in the style
of %e
or %E
(respectively) if the exponent would be less
than -4 or greater than or equal to the precision; otherwise they use
the %f
style. A precision of 0
, is taken as 1. is
Trailing zeros are removed from the fractional portion of the result and
a decimal-point character appears only if it is followed by a digit.
The %a
and %A
conversions are meant for representing
floating-point numbers exactly in textual form so that they can be
exchanged as texts between different programs and/or machines. The
numbers are represented is the form
[-
]0x
h.
hhhp
[+
|-
]dd.
At the left of the decimal-point character exactly one digit is print.
This character is only 0
if the number is denormalized.
Otherwise the value is unspecified; it is implementation dependent how many
bits are used. The number of hexadecimal digits on the right side of
the decimal-point character is equal to the precision. If the precision
is zero it is determined to be large enough to provide an exact
representation of the number (or it is large enough to distinguish two
adjacent values if the FLT_RADIX
is not a power of 2,
see Floating Point Parameters). For the %a
conversion
lower-case characters are used to represent the hexadecimal number and
the prefix and exponent sign are printed as 0x
and p
respectively. Otherwise upper-case characters are used and 0X
and P
are used for the representation of prefix and exponent
string. The exponent to the base of two is printed as a decimal number
using at least one digit but at most as many digits as necessary to
represent the value exactly.
If the value to be printed represents infinity or a NaN, the output is
[-
]inf
or nan
respectively if the conversion
specifier is %a
, %e
, %f
, or %g
and it is
[-
]INF
or NAN
respectively if the conversion is
%A
, %E
, or %G
.
The following flags can be used to modify the behavior:
-
+
+
flag ensures that the result includes
a sign, this flag is ignored if you supply both of them.
#
%g
and %G
conversions,
this also forces trailing zeros after the decimal point to be left
in place where they would otherwise be removed.
'
LC_NUMERIC
category;
see General Numeric. This flag is a GNU extension.
0
-
flag is also
specified.
The precision specifies how many digits follow the decimal-point
character for the %f
, %e
, and %E
conversions. For
these conversions, the default precision is 6
. If the precision
is explicitly 0
, this suppresses the decimal point character
entirely. For the %g
and %G
conversions, the precision
specifies how many significant digits to print. Significant digits are
the first digit before the decimal point, and all the digits after it.
If the precision is 0
or not specified for %g
or %G
,
it is treated like a value of 1
. If the value being printed
cannot be expressed accurately in the specified number of digits, the
value is rounded to the nearest number that fits.
Without a type modifier, the floating-point conversions use an argument
of type double
. (By the default argument promotions, any
float
arguments are automatically converted to double
.)
The following type modifier is supported:
L
L
specifies that the argument is a long
double
.
Here are some examples showing how numbers print using the various
floating-point conversions. All of the numbers were printed using
this template string:
"|%13.4a|%13.4f|%13.4e|%13.4g|\n"
Here is the output:
| 0x0.0000p+0| 0.0000| 0.0000e+00| 0| | 0x1.0000p-1| 0.5000| 5.0000e-01| 0.5| | 0x1.0000p+0| 1.0000| 1.0000e+00| 1| | -0x1.0000p+0| -1.0000| -1.0000e+00| -1| | 0x1.9000p+6| 100.0000| 1.0000e+02| 100| | 0x1.f400p+9| 1000.0000| 1.0000e+03| 1000| | 0x1.3880p+13| 10000.0000| 1.0000e+04| 1e+04| | 0x1.81c8p+13| 12345.0000| 1.2345e+04| 1.234e+04| | 0x1.86a0p+16| 100000.0000| 1.0000e+05| 1e+05| | 0x1.e240p+16| 123456.0000| 1.2346e+05| 1.235e+05|
Notice how the %g
conversion drops trailing zeros.
This section describes miscellaneous conversions for printf
.
The %c
conversion prints a single character. In case there is no
l
modifier the int
argument is first converted to an
unsigned char
. Then, if used in a wide stream function, the
character is converted into the corresponding wide character. The
-
flag can be used to specify left-justification in the field,
but no other flags are defined, and no precision or type modifier can be
given. For example:
printf ("%c%c%c%c%c", 'h', 'e', 'l', 'l', 'o');
prints hello
.
If there is a l
modifier present the argument is expected to be
of type wint_t
. If used in a multibyte function the wide
character is converted into a multibyte character before being added to
the output. In this case more than one output byte can be produced.
The %s
conversion prints a string. If no l
modifier is
present the corresponding argument must be of type char *
(or
const char *
). If used in a wide stream function the string is
first converted in a wide character string. A precision can be
specified to indicate the maximum number of characters to write;
otherwise characters in the string up to but not including the
terminating null character are written to the output stream. The
-
flag can be used to specify left-justification in the field,
but no other flags or type modifiers are defined for this conversion.
For example:
printf ("%3s%-6s", "no", "where");
prints nowhere
.
If there is a l
modifier present the argument is expected to be of type wchar_t
(or const wchar_t *
).
If you accidentally pass a null pointer as the argument for a %s
conversion, the GNU library prints it as (null)
. We think this
is more useful than crashing. But it's not good practice to pass a null
argument intentionally.
The %m
conversion prints the string corresponding to the error
code in errno
. See Error Messages. Thus:
fprintf (stderr, "can't open `%s': %m\n", filename);
is equivalent to:
fprintf (stderr, "can't open `%s': %s\n", filename, strerror (errno));
The %m
conversion is a GNU C library extension.
The %p
conversion prints a pointer value. The corresponding
argument must be of type void *
. In practice, you can use any
type of pointer.
In the GNU system, non-null pointers are printed as unsigned integers,
as if a %#x
conversion were used. Null pointers print as
(nil)
. (Pointers might print differently in other systems.)
For example:
printf ("%p", "testing");
prints 0x
followed by a hexadecimal number--the address of the
string constant "testing"
. It does not print the word
testing
.
You can supply the -
flag with the %p
conversion to
specify left-justification, but no other flags, precision, or type
modifiers are defined.
The %n
conversion is unlike any of the other output conversions.
It uses an argument which must be a pointer to an int
, but
instead of printing anything it stores the number of characters printed
so far by this call at that location. The h
and l
type
modifiers are permitted to specify that the argument is of type
short int *
or long int *
instead of int *
, but no
flags, field width, or precision are permitted.
For example,
int nchar; printf ("%d %s%n\n", 3, "bears", &nchar);
prints:
3 bears
and sets nchar
to 7
, because 3 bears
is seven
characters.
The %%
conversion prints a literal %
character. This
conversion doesn't use an argument, and no flags, field width,
precision, or type modifiers are permitted.
This section describes how to call printf
and related functions.
Prototypes for these functions are in the header file stdio.h
.
Because these functions take a variable number of arguments, you
must declare prototypes for them before using them. Of course,
the easiest way to make sure you have all the right prototypes is to
just include stdio.h
.
int printf (const char *template, ...) | Function |
The printf function prints the optional arguments under the
control of the template string template to the stream
stdout . It returns the number of characters printed, or a
negative value if there was an output error.
|
int wprintf (const wchar_t *template, ...) | Function |
The wprintf function prints the optional arguments under the
control of the wide template string template to the stream
stdout . It returns the number of wide characters printed, or a
negative value if there was an output error.
|
int fprintf (FILE *stream, const char *template, ...) | Function |
This function is just like printf , except that the output is
written to the stream stream instead of stdout .
|
int fwprintf (FILE *stream, const wchar_t *template, ...) | Function |
This function is just like wprintf , except that the output is
written to the stream stream instead of stdout .
|
int sprintf (char *s, const char *template, ...) | Function |
This is like printf , except that the output is stored in the character
array s instead of written to a stream. A null character is written
to mark the end of the string.
The The behavior of this function is undefined if copying takes place
between objects that overlap--for example, if s is also given
as an argument to be printed under control of the Warning: The To avoid this problem, you can use |
int swprintf (wchar_t *s, size_t size, const wchar_t *template, ...) | Function |
This is like wprintf , except that the output is stored in the
wide character array ws instead of written to a stream. A null
wide character is written to mark the end of the string. The size
argument specifies the maximum number of characters to produce. The
trailing null character is counted towards this limit, so you should
allocate at least size wide characters for the string ws.
The return value is the number of characters generated for the given
input, excluding the trailing null. If not all output fits into the
provided buffer a negative value is returned. You should try again with
a bigger output string. Note: this is different from how
Note that the corresponding narrow stream function takes fewer
parameters. |
int snprintf (char *s, size_t size, const char *template, ...) | Function |
The snprintf function is similar to sprintf , except that
the size argument specifies the maximum number of characters to
produce. The trailing null character is counted towards this limit, so
you should allocate at least size characters for the string s.
The return value is the number of characters which would be generated
for the given input, excluding the trailing null. If this value is
greater or equal to size, not all characters from the result have
been stored in s. You should try again with a bigger output
string. Here is an example of doing this:
/* Construct a message describing the value of a variable whose name is name and whose value is value. */ char * make_message (char *name, char *value) { /* Guess we need no more than 100 chars of space. */ int size = 100; char *buffer = (char *) xmalloc (size); int nchars; if (buffer == NULL) return NULL; /* Try to print in the allocated space. */ nchars = snprintf (buffer, size, "value of %s is %s", name, value); if (nchars >= size) { /* Reallocate buffer now that we know how much space is needed. */ buffer = (char *) xrealloc (buffer, nchars + 1); if (buffer != NULL) /* Try again. */ snprintf (buffer, size, "value of %s is %s", name, value); } /* The last call worked, return the string. */ return buffer; } In practice, it is often easier just to use Attention: In versions of the GNU C library prior to 2.1 the
return value is the number of characters stored, not including the
terminating null; unless there was not enough space in s to
store the result in which case |
The functions in this section do formatted output and place the results in dynamically allocated memory.
int asprintf (char **ptr, const char *template, ...) | Function |
This function is similar to sprintf , except that it dynamically
allocates a string (as with malloc ; see Unconstrained Allocation) to hold the output, instead of putting the output in a
buffer you allocate in advance. The ptr argument should be the
address of a char * object, and asprintf stores a pointer
to the newly allocated string at that location.
The return value is the number of characters allocated for the buffer, or less than zero if an error occurred. Usually this means that the buffer could not be allocated. Here is how to use /* Construct a message describing the value of a variable whose name is name and whose value is value. */ char * make_message (char *name, char *value) { char *result; if (asprintf (&result, "value of %s is %s", name, value) < 0) return NULL; return result; } |
int obstack_printf (struct obstack *obstack, const char *template, ...) | Function |
This function is similar to asprintf , except that it uses the
obstack obstack to allocate the space. See Obstacks.
The characters are written onto the end of the current object.
To get at them, you must finish the object with |
The functions vprintf
and friends are provided so that you can
define your own variadic printf
-like functions that make use of
the same internals as the built-in formatted output functions.
The most natural way to define such functions would be to use a language
construct to say, "Call printf
and pass this template plus all
of my arguments after the first five." But there is no way to do this
in C, and it would be hard to provide a way, since at the C language
level there is no way to tell how many arguments your function received.
Since that method is impossible, we provide alternative functions, the
vprintf
series, which lets you pass a va_list
to describe
"all of my arguments after the first five."
When it is sufficient to define a macro rather than a real function,
the GNU C compiler provides a way to do this much more easily with macros.
For example:
#define myprintf(a, b, c, d, e, rest...) \ printf (mytemplate , ## rest...)
See Macros with Variable Numbers of Arguments, for details. But this is limited to macros, and does not apply to real functions at all.
Before calling vprintf
or the other functions listed in this
section, you must call va_start
(see Variadic Functions) to initialize a pointer to the variable arguments. Then you
can call va_arg
to fetch the arguments that you want to handle
yourself. This advances the pointer past those arguments.
Once your va_list
pointer is pointing at the argument of your
choice, you are ready to call vprintf
. That argument and all
subsequent arguments that were passed to your function are used by
vprintf
along with the template that you specified separately.
In some other systems, the va_list
pointer may become invalid
after the call to vprintf
, so you must not use va_arg
after you call vprintf
. Instead, you should call va_end
to retire the pointer from service. However, you can safely call
va_start
on another pointer variable and begin fetching the
arguments again through that pointer. Calling vprintf
does not
destroy the argument list of your function, merely the particular
pointer that you passed to it.
GNU C does not have such restrictions. You can safely continue to fetch
arguments from a va_list
pointer after passing it to
vprintf
, and va_end
is a no-op. (Note, however, that
subsequent va_arg
calls will fetch the same arguments which
vprintf
previously used.)
Prototypes for these functions are declared in stdio.h
.
int vprintf (const char *template, va_list ap) | Function |
This function is similar to printf except that, instead of taking
a variable number of arguments directly, it takes an argument list
pointer ap.
|
int vwprintf (const wchar_t *template, va_list ap) | Function |
This function is similar to wprintf except that, instead of taking
a variable number of arguments directly, it takes an argument list
pointer ap.
|
int vfprintf (FILE *stream, const char *template, va_list ap) | Function |
This is the equivalent of fprintf with the variable argument list
specified directly as for vprintf .
|
int vfwprintf (FILE *stream, const wchar_t *template, va_list ap) | Function |
This is the equivalent of fwprintf with the variable argument list
specified directly as for vwprintf .
|
int vsprintf (char *s, const char *template, va_list ap) | Function |
This is the equivalent of sprintf with the variable argument list
specified directly as for vprintf .
|
int vswprintf (wchar_t *s, size_t size, const wchar_t *template, va_list ap) | Function |
This is the equivalent of swprintf with the variable argument list
specified directly as for vwprintf .
|
int vsnprintf (char *s, size_t size, const char *template, va_list ap) | Function |
This is the equivalent of snprintf with the variable argument list
specified directly as for vprintf .
|
int vasprintf (char **ptr, const char *template, va_list ap) | Function |
The vasprintf function is the equivalent of asprintf with the
variable argument list specified directly as for vprintf .
|
int obstack_vprintf (struct obstack *obstack, const char *template, va_list ap) | Function |
The obstack_vprintf function is the equivalent of
obstack_printf with the variable argument list specified directly
as for vprintf .
|
Here's an example showing how you might use vfprintf
. This is a
function that prints error messages to the stream stderr
, along
with a prefix indicating the name of the program
(see Error Messages, for a description of
program_invocation_short_name
).
#include <stdio.h> #include <stdarg.h> void eprintf (const char *template, ...) { va_list ap; extern char *program_invocation_short_name; fprintf (stderr, "%s: ", program_invocation_short_name); va_start (ap, template); vfprintf (stderr, template, ap); va_end (ap); }
You could call eprintf
like this:
eprintf ("file `%s' does not exist\n", filename);
In GNU C, there is a special construct you can use to let the compiler
know that a function uses a printf
-style format string. Then it
can check the number and types of arguments in each call to the
function, and warn you when they do not match the format string.
For example, take this declaration of eprintf
:
void eprintf (const char *template, ...) __attribute__ ((format (printf, 1, 2)));
This tells the compiler that eprintf
uses a format string like
printf
(as opposed to scanf
; see Formatted Input);
the format string appears as the first argument;
and the arguments to satisfy the format begin with the second.
See Declaring Attributes of Functions, for more information.
You can use the function parse_printf_format
to obtain
information about the number and types of arguments that are expected by
a given template string. This function permits interpreters that
provide interfaces to printf
to avoid passing along invalid
arguments from the user's program, which could cause a crash.
All the symbols described in this section are declared in the header
file printf.h
.
size_t parse_printf_format (const char *template, size_t n, int *argtypes) | Function |
This function returns information about the number and types of
arguments expected by the printf template string template.
The information is stored in the array argtypes; each element of
this array describes one argument. This information is encoded using
the various PA_ macros, listed below.
The argument n specifies the number of elements in the array
argtypes. This is the maximum number of elements that
|
The argument types are encoded as a combination of a basic type and modifier flag bits.
int PA_FLAG_MASK | Macro |
This macro is a bitmask for the type modifier flag bits. You can write
the expression (argtypes[i] & PA_FLAG_MASK) to extract just the
flag bits for an argument, or (argtypes[i] & ~PA_FLAG_MASK) to
extract just the basic type code.
|
Here are symbolic constants that represent the basic types; they stand for integer values.
PA_INT
int
.
PA_CHAR
int
, cast to char
.
PA_STRING
char *
, a null-terminated string.
PA_POINTER
void *
, an arbitrary pointer.
PA_FLOAT
float
.
PA_DOUBLE
double
.
PA_LAST
PA_LAST
. For example, if you have data types foo
and bar
with their own specialized printf
conversions,
you could define encodings for these types as:
#define PA_FOO PA_LAST #define PA_BAR (PA_LAST + 1)
Here are the flag bits that modify a basic type. They are combined with the code for the basic type using inclusive-or.
PA_FLAG_PTR
PA_INT|PA_FLAG_PTR
represents the type int *
.
PA_FLAG_SHORT
short
. (This corresponds to the h
type modifier.)
PA_FLAG_LONG
long
. (This corresponds to the l
type modifier.)
PA_FLAG_LONG_LONG
long long
. (This corresponds to the L
type modifier.)
PA_FLAG_LONG_DOUBLE
PA_FLAG_LONG_LONG
, used by convention with
a base type of PA_DOUBLE
to indicate a type of long double
.
Here is an example of decoding argument types for a format string. We
assume this is part of an interpreter which contains arguments of type
NUMBER
, CHAR
, STRING
and STRUCTURE
(and
perhaps others which are not valid here).
/* Test whether the nargs specified objects in the vector args are valid for the format string format: if so, return 1. If not, return 0 after printing an error message. */ int validate_args (char *format, int nargs, OBJECT *args) { int *argtypes; int nwanted; /* Get the information about the arguments. Each conversion specification must be at least two characters long, so there cannot be more specifications than half the length of the string. */ argtypes = (int *) alloca (strlen (format) / 2 * sizeof (int)); nwanted = parse_printf_format (string, nelts, argtypes); /* Check the number of arguments. */ if (nwanted > nargs) { error ("too few arguments (at least %d required)", nwanted); return 0; } /* Check the C type wanted for each argument and see if the object given is suitable. */ for (i = 0; i < nwanted; i++) { int wanted; if (argtypes[i] & PA_FLAG_PTR) wanted = STRUCTURE; else switch (argtypes[i] & ~PA_FLAG_MASK) { case PA_INT: case PA_FLOAT: case PA_DOUBLE: wanted = NUMBER; break; case PA_CHAR: wanted = CHAR; break; case PA_STRING: wanted = STRING; break; case PA_POINTER: wanted = STRUCTURE; break; } if (TYPE (args[i]) != wanted) { error ("type mismatch for arg number %d", i); return 0; } } return 1; }
printf
The GNU C library lets you define your own custom conversion specifiers
for printf
template strings, to teach printf
clever ways
to print the important data structures of your program.
The way you do this is by registering the conversion with the function
register_printf_function
; see Registering New Conversions.
One of the arguments you pass to this function is a pointer to a handler
function that produces the actual output; see Defining the Output Handler, for information on how to write this function.
You can also install a function that just returns information about the number and type of arguments expected by the conversion specifier. See Parsing a Template String, for information about this.
The facilities of this section are declared in the header file
printf.h
.
register_printf_function
to register a new output conversion.
register_printf_function
.
printf
handler function.
printf
handlers.
Portability Note: The ability to extend the syntax of
printf
template strings is a GNU extension. ISO standard C has
nothing similar.
The function to register a new output conversion is
register_printf_function
, declared in printf.h
.
int register_printf_function (int spec, printf_function handler-function, printf_arginfo_function arginfo-function) | Function |
This function defines the conversion specifier character spec.
Thus, if spec is 'Y' , it defines the conversion %Y .
You can redefine the built-in conversions like %s , but flag
characters like # and type modifiers like l can never be
used as conversions; calling register_printf_function for those
characters has no effect. It is advisable not to use lowercase letters,
since the ISO C standard warns that additional lowercase letters may be
standardized in future editions of the standard.
The handler-function is the function called by The arginfo-function is the function called by
Attention: In the GNU C library versions before 2.0 the
arginfo-function function did not need to be installed unless
the user used the The return value is You can redefine the standard output conversions, but this is probably not a good idea because of the potential for confusion. Library routines written by other people could break if you do this. |
If you define a meaning for %A
, what if the template contains
%+23A
or %-#A
? To implement a sensible meaning for these,
the handler when called needs to be able to get the options specified in
the template.
Both the handler-function and arginfo-function accept an
argument that points to a struct printf_info
, which contains
information about the options appearing in an instance of the conversion
specifier. This data type is declared in the header file
printf.h
.
struct printf_info | Type |
This structure is used to pass information about the options appearing
in an instance of a conversion specifier in a printf template
string to the handler and arginfo functions for that specifier. It
contains the following members:
|
Now let's look at how to define the handler and arginfo functions
which are passed as arguments to register_printf_function
.
Compatibility Note: The interface changed in GNU libc
version 2.0. Previously the third argument was of type
va_list *
.
You should define your handler functions with a prototype like:
int function (FILE *stream, const struct printf_info *info, const void *const *args)
The stream argument passed to the handler function is the stream to which it should write output.
The info argument is a pointer to a structure that contains information about the various options that were included with the conversion in the template string. You should not modify this structure inside your handler function. See Conversion Specifier Options, for a description of this data structure.
The args is a vector of pointers to the arguments data. The number of arguments was determined by calling the argument information function provided by the user.
Your handler function should return a value just like printf
does: it should return the number of characters it has written, or a
negative value to indicate an error.
printf_function | Data Type |
This is the data type that a handler function should have. |
If you are going to use parse_printf_format
in your
application, you must also define a function to pass as the
arginfo-function argument for each new conversion you install with
register_printf_function
.
You have to define these functions with a prototype like:
int function (const struct printf_info *info, size_t n, int *argtypes)
The return value from the function should be the number of arguments the
conversion expects. The function should also fill in no more than
n elements of the argtypes array with information about the
types of each of these arguments. This information is encoded using the
various PA_
macros. (You will notice that this is the same
calling convention parse_printf_format
itself uses.)
printf_arginfo_function | Data Type |
This type is used to describe functions that return information about the number and type of arguments used by a conversion specifier. |
printf
Extension ExampleHere is an example showing how to define a printf
handler function.
This program defines a data structure called a Widget
and
defines the %W
conversion to print information about Widget *
arguments, including the pointer value and the name stored in the data
structure. The %W
conversion supports the minimum field width and
left-justification options, but ignores everything else.
#include <stdio.h> #include <stdlib.h> #include <printf.h> typedef struct { char *name; } Widget; int print_widget (FILE *stream, const struct printf_info *info, const void *const *args) { const Widget *w; char *buffer; int len; /* Format the output into a string. */ w = *((const Widget **) (args[0])); len = asprintf (&buffer, "<Widget %p: %s>", w, w->name); if (len == -1) return -1; /* Pad to the minimum field width and print to the stream. */ len = fprintf (stream, "%*s", (info->left ? -info->width : info->width), buffer); /* Clean up and return. */ free (buffer); return len; } int print_widget_arginfo (const struct printf_info *info, size_t n, int *argtypes) { /* We always take exactly one argument and this is a pointer to the structure.. */ if (n > 0) argtypes[0] = PA_POINTER; return 1; } int main (void) { /* Make a widget to print. */ Widget mywidget; mywidget.name = "mywidget"; /* Register the print function for widgets. */ register_printf_function ('W', print_widget, print_widget_arginfo); /* Now print the widget. */ printf ("|%W|\n", &mywidget); printf ("|%35W|\n", &mywidget); printf ("|%-35W|\n", &mywidget); return 0; }
The output produced by this program looks like:
|<Widget 0xffeffb7c: mywidget>| | <Widget 0xffeffb7c: mywidget>| |<Widget 0xffeffb7c: mywidget> |
printf
HandlersThe GNU libc also contains a concrete and useful application of the
printf
handler extension. There are two functions available
which implement a special way to print floating-point numbers.
int printf_size (FILE *fp, const struct printf_info *info, const void *const *args) | Function |
Print a given floating point number as for the format %f except
that there is a postfix character indicating the divisor for the
number to make this less than 1000. There are two possible divisors:
powers of 1024 or powers of 1000. Which one is used depends on the
format character specified while registered this handler. If the
character is of lower case, 1024 is used. For upper case characters,
1000 is used.
The postfix tag corresponds to bytes, kilobytes, megabytes, gigabytes, etc. The full table is: The default precision is 3, i.e., 1024 is printed with a lower-case
format character as if it were |
Due to the requirements of register_printf_function
we must also
provide the function which returns information about the arguments.
int printf_size_info (const struct printf_info *info, size_t n, int *argtypes) | Function |
This function will return in argtypes the information about the
used parameters in the way the vfprintf implementation expects
it. The format always takes one argument.
|
To use these functions both functions must be registered with a call like
register_printf_function ('B', printf_size, printf_size_info);
Here we register the functions to print numbers as powers of 1000 since
the format character 'B'
is an upper-case character. If we
would additionally use 'b'
in a line like
register_printf_function ('b', printf_size, printf_size_info);
we could also print using a power of 1024. Please note that all that is
different in these two lines is the format specifier. The
printf_size
function knows about the difference between lower and upper
case format specifiers.
The use of 'B'
and 'b'
is no coincidence. Rather it is
the preferred way to use this functionality since it is available on
some other systems which also use format specifiers.
The functions described in this section (scanf
and related
functions) provide facilities for formatted input analogous to the
formatted output facilities. These functions provide a mechanism for
reading arbitrary values under the control of a format string or
template string.
malloc
the buffer.
vscanf
and friends.
Calls to scanf
are superficially similar to calls to
printf
in that arbitrary arguments are read under the control of
a template string. While the syntax of the conversion specifications in
the template is very similar to that for printf
, the
interpretation of the template is oriented more towards free-format
input and simple pattern matching, rather than fixed-field formatting.
For example, most scanf
conversions skip over any amount of
"white space" (including spaces, tabs, and newlines) in the input
file, and there is no concept of precision for the numeric input
conversions as there is for the corresponding output conversions.
Ordinarily, non-whitespace characters in the template are expected to
match characters in the input stream exactly, but a matching failure is
distinct from an input error on the stream.
Another area of difference between scanf
and printf
is
that you must remember to supply pointers rather than immediate values
as the optional arguments to scanf
; the values that are read are
stored in the objects that the pointers point to. Even experienced
programmers tend to forget this occasionally, so if your program is
getting strange errors that seem to be related to scanf
, you
might want to double-check this.
When a matching failure occurs, scanf
returns immediately,
leaving the first non-matching character as the next character to be
read from the stream. The normal return value from scanf
is the
number of values that were assigned, so you can use this to determine if
a matching error happened before all the expected values were read.
The scanf
function is typically used for things like reading in
the contents of tables. For example, here is a function that uses
scanf
to initialize an array of double
:
void readarray (double *array, int n) { int i; for (i=0; i<n; i++) if (scanf (" %lf", &(array[i])) != 1) invalid_input_error (); }
The formatted input functions are not used as frequently as the formatted output functions. Partly, this is because it takes some care to use them properly. Another reason is that it is difficult to recover from a matching error.
If you are trying to read input that doesn't match a single, fixed
pattern, you may be better off using a tool such as Flex to generate a
lexical scanner, or Bison to generate a parser, rather than using
scanf
. For more information about these tools, see Top, and Top.
A scanf
template string is a string that contains ordinary
multibyte characters interspersed with conversion specifications that
start with %
.
Any whitespace character (as defined by the isspace
function;
see Classification of Characters) in the template causes any number
of whitespace characters in the input stream to be read and discarded.
The whitespace characters that are matched need not be exactly the same
whitespace characters that appear in the template string. For example,
write ,
in the template to recognize a comma with optional
whitespace before and after.
Other characters in the template string that are not part of conversion specifications must match characters in the input stream exactly; if this is not the case, a matching failure occurs.
The conversion specifications in a scanf
templa