2 * Copyright (C) 2013 Red Hat Inc.
4 * This program is free software; you can redistribute it and/or modify
5 * it under the terms of the GNU General Public License as published by
6 * the Free Software Foundation; either version 2 of the License, or
7 * (at your option) any later version.
9 * This program is distributed in the hope that it will be useful,
10 * but WITHOUT ANY WARRANTY; without even the implied warranty of
11 * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
12 * GNU General Public License for more details.
14 * You should have received a copy of the GNU General Public License along
15 * with this program; if not, write to the Free Software Foundation, Inc.,
16 * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
19 (** {1 Goaljobs library of useful helper functions.} *)
21 (** {2 Target and require}
23 These are used to write goals.
25 Normally you write a goal with zero or one [target] and
26 zero or more [require]s, as the examples below should make
29 In the first example, the target is that the [o_file] (object) exists
30 and is newer than the [c_file] (source). The goal
31 meets that target by running the C compiler ([cc]) which, if it
32 succeeds, will ensure that the object file exists and is newer
36 let goal compiled c_file =
37 let o_file = change_file_extension "o" c_file in
38 target (more_recent [o_file] [c_file]);
40 sh "cd $builddir && cc -c %s -o %s" c_file o_file
43 In the second example, the goal requires that several files
44 have been compiled ([require (compiled ...)])
45 before it can link the final program:
48 let goal built program source =
49 target (more_recent [program] [source]);
51 require (compiled source);
53 let object = change_file_extension "o" source in
54 sh "cd $builddir && cc %s -o %s" object program
59 val target : bool -> unit
60 (** [target] {i condition} defines the target condition that {b will}
61 be met once the current goal has run.
63 You can think of the target as a promise or contract that you
64 make, which is met by running the rest of the goal.
66 Goaljobs is much more flexible than [make]. In [make] only a
67 single type of target is possible. The following are roughly
76 let goal compiled () =
77 target (more_recent ["foo.o"] ["foo.c"]);
78 require (file_exists "foo.c");
82 Targets in goaljobs can be any arbitrary expression. For
83 example, it can access network resources or test URLs.
85 Almost every goal should have one target, which should
86 accurately state the outcome once the goal has been run.
88 It is possible to have no target. This means the goal
89 always runs (like using "force" in make).
91 You should not have multiple targets in a single goal. They
92 won't work the way you expect, and future versions of goaljobs
93 will likely stop you from doing this.
95 Normally you put the target(s) early on in the goal, before any
96 running code and before any [require]s. This is not a
97 hard-and-fast rule and it is not enforced, but doing it will
98 ensure the goal runs most efficiently since if the target is met
99 already then the rest of the goal doesn't run. *)
101 val target_all : bool list -> unit
102 (** [target_all [t1; t2; ...]] is the same as writing
103 [target (t1 && t2 && ...)] *)
105 val target_exists : bool list -> unit
106 (** [target_exists [t1; t2; ...]] is the same as writing
107 [target (t1 || t2 || ...)] *)
109 val require : (unit -> unit) -> unit
110 (** [require] {i goal} defines the requirements of this goal, that
111 is, other goals that have to be met before the rest of the
114 In terms of [make], [require]s are roughly equivalent to the
115 right hand side after the [:], but in goaljobs the requirements
116 can be much richer than simply "that file must exist".
118 Some very simple goals don't need any [require]s. You can
119 have as many [require]s as you need in a goal, and you can
120 use a loop or make them conditional if you want.
122 Unlike [make], the requirements of a goal can be
123 placed anywhere within the goal, as long as you put them
124 before they are needed. *)
126 (** {2 Periodic jobs}
128 If you want to have a goal that runs when some outside event
129 happens you have three choices: Manually run the script (this is
130 basically what [make] forces you to do). Have some sort of hook
131 that runs the script (eg. a git hook). Or use a periodic job to
132 poll for an event or change.
134 Periodic jobs run regularly to poll for an outside event or
135 change. If a script has periodic jobs, then it runs continuously
136 (or until you kill it).
138 An example of a script that checks for new git commits and when
139 it sees one it will ensure it passes the tests:
142 let repo = Sys.getenv "HOME" // "repo"
144 let goal git_commit_tested commit =
145 let key = sprintf "repo-tested-%s" commit in
146 target (memory_exists key);
156 (* Record that this commit was tested successfully. *)
159 every 30 minutes (fun () ->
160 let commit = shout "cd %s && git rev-parse HEAD" repo in
161 (* Require that this commit has been tested. *)
162 require (git_commit_tested commit)
166 Some notes about the above example: Firstly only the current HEAD
167 commit is required to be tested. This is because older commits
168 are irrelevant and because if they failed the test before there is
169 not point retesting them (commits are immutable). Secondly we use
170 the Memory to remember that we have successfully tested a commit.
171 This is what stops the program from repeatedly testing the same
174 (* This is what lets you write '30 minutes' etc: *)
175 type period_t = Seconds | Days | Months | Years
176 val seconds : int * period_t
177 val sec : int * period_t
178 val secs : int * period_t
179 val second : int * period_t
180 val minutes : int * period_t
181 val min : int * period_t
182 val mins : int * period_t
183 val minute : int * period_t
184 val hours : int * period_t
185 val hour : int * period_t
186 val days : int * period_t
187 val day : int * period_t
188 val weeks : int * period_t
189 val week : int * period_t
190 val months : int * period_t
191 val month : int * period_t
192 val years : int * period_t
193 val year : int * period_t
195 val every : ?name:string -> int -> int * period_t -> (unit -> unit) -> unit
196 (** [every N (seconds|minutes|hours|days|weeks|months|years) f]
197 runs the function [f] periodically.
199 The optional [~name] parameter can be used to name the job
202 (** {2 File and URL testing}
204 Various functions to test the existence of files, URLs.
207 val file_exists : string -> bool
208 (** Return true if the named file exists.
210 This function also exists as a goal. Writing:
211 {v require (file_exists "somefile"); v}
212 will die unless ["somefile"] exists. *)
214 val directory_exists : string -> bool
215 (** Return true if the named directory exists.
217 There is also a goal version of this function. *)
219 val file_newer_than : string -> string -> bool
220 (** [file_newer_than file_a file_b] returns true if [file_a] is
221 newer than [file_b]. Note that if [file_a] does not exist, it
222 returns false. If [file_b] does not exist, it is an error.
224 There is also a goal version of this function. *)
226 val more_recent : string list -> string list -> bool
227 (** [more_recent objects sources] expresses the [make] relationship:
229 {v object(s) ...: source(s) ... v}
234 let goal built objects sources =
235 target (more_recent objects sources);
236 ... code to rebuild ...
239 It is roughly equivalent to checking that all the object files
240 exist and are newer than all of the source files.
242 Note that both parameters are lists (since in [make] you can
243 have a list of source files and a list of object files). If you
244 don't want a list, pass a single-element list containing the
245 single the object/source file.
247 There is also a goal version of this function. *)
249 val url_exists : string -> bool
250 (** The URL is tested to see if it exists.
252 There is also a goal version of this function. *)
254 val file_contains_string : string -> string -> bool
255 (** [file_contains_string filename str] checks if the named file
256 contains the given substring [str].
258 There is also a goal version of this function. *)
260 val url_contains_string : string -> string -> bool
261 (** [url_contains_string url str] downloads the URL and checks
262 whether the content contains the given substring [str].
264 There is also a goal version of this function. *)
266 val (//) : string -> string -> string
267 (** Concatenate two paths. *)
269 val quote : string -> string
270 (** Quote the string to make it safe to pass directly to the shell. *)
274 Call out to the Unix shell. [/bin/sh] is used unless you set
275 {!shell} to some other value. Note that the environment variable
276 [SHELL] is {i not} used.
278 {!sh}, {!shout}, {!shlines} work like [printf]. ie. You can
279 substitute variables using [%s], [%d] and so on. For example:
282 sh "rsync foo-%s.tar.gz example.com:/html/" version
285 Each invocation of {!sh} (etc) is a single shell (this is slightly
286 different from how [make] works). For example:
291 tarball=$package.tar.gz
300 The shell error mode is set such that if any single command
301 returns an error then the {!sh} function as a whole exits with
304 to ignore the result of a command.
306 Each shell runs in a new temporary directory. The temporary
307 directory and all its contents is deleted after the shell exits.
308 If you want to save any data, [cd] somewhere. The environment
309 variable [$builddir] is exported to the script. This is the
310 current directory when the goaljobs program was started.
312 For example you could start the command sequence with
313 [cd $HOME/data/] or [cd $builddir].
316 val sh : ('a, unit, string, unit) format4 -> 'a
317 (** Run the command(s). *)
319 val shout : ('a, unit, string, string) format4 -> 'a
320 (** Run the command(s).
322 Anything printed on stdout is returned as a string.
323 The trailing [\n] character, if any, is not returned. *)
325 val shlines : ('a, unit, string, string list) format4 -> 'a
326 (** Run the command(s).
328 Any lines printed to stdout is returned as a list of strings.
329 Trailing [\n] characters are not returned. *)
331 val shell : string ref
332 (** Set this variable to override the default shell ([/bin/sh]). *)
334 (** {2 String functions}
336 Most string functions are provided by the OCaml standard
337 library (see the module [String]). For convenience some
338 extra functions are provided here. *)
341 val replace_substring : string -> string -> string -> string
342 (** [replace_substring patt repl string] replaces all occurrences
343 of [patt] with [repl] in [string]. *)
346 val change_file_extension : string -> string -> string
347 (** [change_file_extension ext filename] changes the file extension
348 of [filename] to [.ext]. For example
349 [change_file_extension "o" "main.c"] returns ["main.o"].
350 If the original filename has no extension, this function
351 adds the extension. *)
354 val filter_file_extension : string -> string list -> string
355 (** [filter_file_extension ext filenames] returns only those
356 filenames in the list which have the given file extension.
357 For example [filter_file_extension "o" ["foo.c"; "bar.o"]]
358 would return [["bar.o"]] (a single element list). *)
361 (** {2 Memory (persistent key/value storage)}
363 "The Memory" is key/value storage which persists across goaljobs
364 sessions. It is stored in the file [$HOME/.goaljobs-memory]
365 (which is a binary file, but you can delete it if you want).
367 The Memory is locked during accesses, so it is safe to read
368 or write it from multiple parallel goaljobs sessions.
370 Keys and values are strings. The keys should be globally
371 unique, so it is suggested you use some application-specific
372 prefix. eg: "myapp-key"
377 let goal tested version =
378 let key = "myapp-tested-" ^ version in
379 target (memory_exists key);
381 ... some work to test version ...
386 Note in that example the value ["1"] is arbitrary. You just
387 want to store {i any} value so that a later call to {!memory_exists}
390 val memory_exists : string -> bool
391 (** [memory_exists key] checks that the named [key] exists in
392 the Memory. It doesn't matter what value it has.
394 This is also available as a goal, so you can write
395 [require (memory_exists key)] *)
397 val memory_set : string -> string -> unit
398 (** Set [key] to [value] in the Memory. *)
400 val memory_get : string -> string option
401 (** Return the current value of [key] in the Memory. Returns [None]
402 if the key has never been set or was deleted. *)
404 val memory_delete : string -> unit
405 (** Delete the [key]. If the key doesn't exist, has no effect. *)
407 (** {2 Publishing goals}
409 To "publish" a goal means it's available on the command line
410 for users to use directly.
412 Goals that have zero arguments are {b automatically published}.
416 let goal clean () = sh "rm *~"
419 can be used on the command line:
423 The special goal called [all] (if it exists) is run implicitly
424 unless the user specifies another goal. Unlike [make], there is
425 nothing special about the first goal in the file.
427 You can also publish goals, especially ones which take a non-zero
428 number of parameters, by calling {!publish}.
431 val publish : string -> (string list -> unit) -> unit
432 (** Publish the named goal.
434 Use this function as in this example:
437 let goal compiled program sources =
438 ... stuff for building the program from sources ...
440 let () = publish "compiled" (
442 let program = List.hd args in
443 let sources = List.tl args in
444 require (compiled program sources)
448 This could be used as follows:
450 {v ./script compiled program main.c utils.c v}
452 You will notice you have to write a bit of OCaml code to
453 map the string arguments from the command line on to the
454 goal arguments. In the example it means taking the first
455 string argument as the program name, and the rest of the
456 string arguments as the source filenames. This is also
457 the place to perform string to int conversion, checks, and
458 so on (remember that OCaml is strongly typed). *)
462 (* Goal versions of some common functions. You are using these
463 * versions when you write something like:
464 * require (file_exists "foo");
465 * They work the same way as the regular function, except they die
466 * if the predicate returns false.
468 val goal_file_exists : string -> unit
469 val goal_directory_exists : string -> unit
470 val goal_file_newer_than : string -> string -> unit
471 val goal_more_recent : string list -> string list -> unit
472 val goal_url_exists : string -> unit
473 val goal_file_contains_string : string -> string -> unit
474 val goal_url_contains_string : string -> string -> unit
475 val goal_memory_exists : string -> unit
477 (* A single call to this function is added by the 'goaljobs' script.
478 * It is responsible for parsing the command line and so on.
480 val init : unit -> unit
482 (* Export this so the macros can catch these exceptions. *)
483 type goal_result_t = Goal_OK | Goal_failed of string
484 exception Goal_result of goal_result_t