5 goaljobs-reference - reference documentation for writing goaljobs scripts
13 let goal name args... =
17 (* code to implement the goal *)
20 require (name args...)
29 Goaljobs is a flexible build system and business rules manager similar
30 to make and cron, but much more powerful. You can use it to automate
31 many complex tasks that have multiple steps (even with manual steps)
32 that have to be carried out in dependency order.
34 For a tutorial-like introduction to goaljobs, see
35 L<http://rwmj.wordpress.com/tag/goaljobs/>
37 For examples, see the C<examples/> directory in the source and
38 L<http://git.annexia.org/?p=goals.git;a=summary>
40 For reference documentation on how to write scripts, see below.
42 Note this man page does not cover the whole Goaljobs API. To read
43 about the Goaljobs API, look for the file C<goaljobs.mli> (in the
44 source code or installed as part of the goaljobs package), or see the
45 HTML documentation installed as part of the goaljobs package.
47 =head1 THE SCRIPT FILE
49 The script file should usually start with opening these modules (none
50 of this are required, it's just useful to have them open):
56 This is followed by goals and/or functions and/or top-level OCaml
57 statements and/or C<every> statements (a.k.a periodic jobs).
59 You can use multiple script files to make up a goaljobs program. You
60 have to list them in dependency order on the goaljobs command line
61 (earlier files required by later files), the same way that the OCaml
62 compiler works. So usually you end up writing:
64 goaljobs utils.ml another_library.ml script.ml
66 where C<script.ml> requires the functions from the utils/library.
67 Note that circular dependencies are not possible.
71 Each goal should have the following basic form:
73 let goal name args... =
77 (* code to implement the goal *)
79 There is no hard-and-fast rule about this. In particular you can put
80 arbitrary OCaml statements anywhere inside a goal (since a goal is
81 just a special form of OCaml function), but sticking to this overall
84 There should be zero or one target. Multiple target statements should
85 not be used in a goal. The target should come as early as possible,
86 and the target condition should be as simple and fast to evaluate as
87 is practical (see L</THE MEMORY> below).
89 There should be zero or any number of C<require> statements. Each
90 require statement should name a single goal (with optional parameters
93 After that should come the code that implements the goal, which might
94 be, for example, a series of shell commands, but could even be
97 As with ordinary OCaml functions, you can define goals recursively
98 or with mutual recursion using:
100 let rec goal1 args... =
107 A goal can also have no arguments:
112 This defines the common goal called C<all>, which acts the same way as
113 C<make all>, ie. if you run the program without any arguments, it will
114 run the C<all> goal if one exists.
116 =head2 PUBLISHING GOALS
118 If a goal is "published" it means it is available to be run directly
119 from the command line. All no-arg goals are published by default.
120 You do not need to do anything special for them. For example:
122 let goal clean () = sh "rm *~"
124 can be used on the command line:
128 For goals which take any parameters, you have to define a small code
129 snippet that converts command line arguments to goal parameters (the
130 reason has to do with OCaml being strongly typed, and because goal
131 parameters might not all be strings).
133 let goal compile program sources =
134 target (more_recent [program] sources);
140 let program = List.hd args in
141 let sources = List.tl args in
142 require (compiled program sources)
147 ./myscript compile program main.c utils.c
149 =head1 TARGET AND REQUIRE
151 The target is promise or contract that you make that the given
152 condition I<will> be true when the goal has finished running.
154 In the first example, the target is that the C<o_file> (object) exists
155 and is newer than the C<c_file> (source). The goal meets that target
156 by running the C compiler (C<cc>) which, if it succeeds, will ensure
157 that the object file exists and is newer than the source file.
159 let goal compiled c_file =
160 let o_file = change_file_extension "o" c_file in
161 target (more_recent [o_file] [c_file]);
168 In the second example, the goal requires that several files have been
169 compiled (C<require (compiled ...)>) before it can link the final
172 let goal built program source =
173 target (more_recent [program] [source]);
175 require (compiled source);
177 let object = change_file_extension "o" source in
183 =head1 SPECIAL VALUES INSIDE GOALS
187 Inside goals, you can use C<goalname> to get the name of the goal, ie:
190 printf "my name is %s\n" goalname
196 =head2 onfail, onsuccess, onrun
198 Inside goals you can register function(s) which run if the goal
199 completes successfully (C<onsuccess>), if the goal completes
200 successfully after running to the end (C<onrun>), or if the goal fails
206 onfail (fun _ -> eprintf "goal '%s' failed\n" goalname);
211 If the shell command (or another part of the goal) fails, then this
216 The single parameter passed to C<onfail> is the exception that was
219 Note that the helper function C<Goaljobs.mailto> is a useful function
220 to call from an C<onfail> handler:
222 let from = "me@example.com"
223 let to_ = "you@example.com"
224 let logfile = log_program_output ()
228 let subject = sprintf "goal: %s: BUILD FAILED" goalname in
229 mailto ~from ~subject ~attach:[logfile] to_);
234 C<onsuccess> and C<onrun> are slightly different from C<onfail> and
237 C<onsuccess> functions can be called if a C<target> condition is met
238 and the rest of the goal is short-circuited. C<onrun> will only be
239 called if all the instructions in the goal actually run and succeed.
241 The single unit C<()> parameter is passed to the C<onsuccess> and
244 You can register as many functions as you want for each handler. The
245 order in which the functions are called is not defined.
249 If you want to have a goal that runs when some outside event happens
250 you have three choices: Manually run the script (this is basically
251 what C<make> forces you to do). Have some sort of hook that runs the
252 script (eg. a git hook). Or use a periodic job to poll for an event
255 Periodic jobs run regularly to poll for an outside event or change.
256 If a script has periodic jobs, then it runs continuously (or until you
259 An example of a script that checks for new git commits and when it
260 sees one it will ensure it passes the tests:
262 let repo = Sys.getenv "HOME" // "repo"
264 let goal git_commit_tested commit =
265 let key = sprintf "repo-tested-%s" commit in
266 target (memory_exists key);
276 (* Record that this commit was tested successfully. *)
279 every 30 minutes (fun () ->
280 let commit = shout "cd %s && git rev-parse HEAD" repo in
281 (* Require that this commit has been tested. *)
282 require (git_commit_tested commit)
285 Some notes about the above example: Firstly only the current HEAD
286 commit is required to be tested. This is because older commits are
287 irrelevant and because if they failed the test before there is not
288 point retesting them (commits are immutable). Secondly we use the
289 Memory to remember that we have successfully tested a commit. This is
290 what stops the program from repeatedly testing the same commit.
294 You can call out to the Unix shell using one of the functions
295 C<Goaljobs.sh>, C<Goaljobs.shout> or C<Goaljobs.shlines>. (These
296 functions are documented in the C<goaljobs.mli> file / API
299 C<sh> runs the command(s). C<shout> collects the output of the
300 command (to stdout only) and returns it as a single string.
301 C<shlines> collects the output and returns it as a list of lines.
303 C<sh>, C<shout>, C<shlines> work like printf. ie. You can substitute
304 variables using C<%s>, C<%d> and so on. For example:
306 sh "rsync foo-%s.tar.gz example.com:/html/" version
308 Each shell runs in a new temporary directory. The temporary directory
309 and all its contents is deleted after the shell exits. If you want to
310 save any data, C<cd> somewhere. If you don't want the temporary
311 directory creation, use C<~tmpdir:false>.
313 The environment variable C<$builddir> is exported to the script. This
314 is the current directory when the goaljobs program was started.
316 Each invocation of C<sh> (etc) is a single shell (this is slightly
317 different from how C<make> works). For example:
321 tarball=$package.tar.gz
329 The shell error mode is set such that if any single command
330 returns an error then the C<sh> function as a whole exits with
335 to ignore the result of a command.
337 C</bin/sh> is used unless you set C<Goaljobs.shell> to some other
338 value. Note that the environment variable C<SHELL> is I<never> used.
342 "The Memory" is key/value storage which persists across goaljobs
343 sessions. It is stored in the file C<$HOME/.goaljobs-memory> (which is
344 a binary file, but you can delete it if you want).
346 The Memory is locked during accesses, so it is safe to read or write
347 it from multiple parallel goaljobs sessions.
349 Keys and values are strings. The keys should be globally unique, so
350 it is suggested you use some application-specific prefix. eg:
355 let goal tested version =
356 let key = "myapp-tested-" ^ version in
357 target (memory_exists key);
359 (* some code to test this version *)
363 Note in that example the value C<1> is arbitrary. You just want to
364 store I<any> value so that a later call to C<memory_exists> will
367 For information about C<Goaljobs.memory_*> APIs see the
368 C<goaljobs.mli> file / API documentation.
376 This is the default shell used by C<sh*> APIs. You can change
377 the shell by setting the C<Goaljobs.shell> reference.
381 The curl program (on the path) is used to check for and download
382 URLs by APIs such as C<Goaljobs.url_exists>.
384 =item C<~/.goaljobs-memory>
386 Persistent key/value store used when you use the C<Goaljobs.memory_*>
391 =head1 ENVIRONMENT VARIABLES
397 This environment variable is set to the current directory when the
398 goals program starts, and is available in goals, shell scripts, etc.
408 Richard W.M. Jones <rjones @ redhat . com>
412 (C) Copyright 2013 Red Hat Inc.,
414 This program is free software; you can redistribute it and/or modify
415 it under the terms of the GNU General Public License as published by
416 the Free Software Foundation; either version 2 of the License, or
417 (at your option) any later version.
419 This program is distributed in the hope that it will be useful,
420 but WITHOUT ANY WARRANTY; without even the implied warranty of
421 MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
422 GNU General Public License for more details.
424 You should have received a copy of the GNU General Public License
425 along with this program; if not, write to the Free Software
426 Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.