#27476 - Multi-threaded compilation of 'syntax-parameterize' forms crashes - GNU bug report logs
GNU bug report logs -
#27476
Multi-threaded compilation of 'syntax-parameterize' forms crashes
Previous
Next
Package:
guile
Reported by:
Leo Famulari
Date: Sat, 24 Jun 2017 16:33:01 UTC
Severity:
serious
Tags: unreproducible
Merged with
27652
28144
31294
31367
31740
32385
34112
34319
Done:
Ludovic Courtès
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first
unarchive
it, by sending
a message to control AT debbugs.gnu.org, with
unarchive 27476
in the body.
You can then email your comments to 27476 AT debbugs.gnu.org in the normal way.
Toggle
the display of automated, internal messages from the tracker.
View this report as an
mbox folder
status mbox
maintainer mbox
Report forwarded
to
bug-guix
bug#27476
; Package
guix
(Sat, 24 Jun 2017 16:33:01 GMT)
Full text
and
rfc822 format
available.
Acknowledgement sent
to
Leo Famulari
New bug report received and forwarded. Copy sent to
bug-guix
(Sat, 24 Jun 2017 16:33:02 GMT)
Full text
and
rfc822 format
available.
Message #5
received at submit
full text
mbox
):
From:
Leo Famulari
To:
bug-guix
Subject:
Wrong type (expecting mutex)
Date:
Sat, 24 Jun 2017 12:32:12 -0400
Message part 1
(text/plain, inline)]
I just got this from `guix pull`:
guix pull --url=file:///gnu/store/l552m9iavw3amq5c8vaifqlxvw09r2nz-guix-latest.tar.gz
unpacking '/gnu/store/l552m9iavw3amq5c8vaifqlxvw09r2nz-guix-latest.tar.gz'...
updating list of substitutes from '
. 100.0%o'... 0.0%
The following derivation will be built:
/gnu/store/xxy7l4jfjx6n62anfqlw4gbmafypqgrs-guix-latest.drv
updating list of substitutes from '
. 100.0%o'... 0.0%
substitute: updating list of substitutes from '
. 100.0%
copying and compiling to '/gnu/store/r1z6nbkrl99hxppcvcprc8vgbzakv632-guix-latest' with Guile 2.2.2...
loading... 25.4% of 606 filesrandom seed for tests: 1498319917
loading... 99.8% of 606 files
compiling... 99.2% of 606 filesBacktrace:
2 (primitive-load "/gnu/store/v9a9cqzh41qg4sixl2mk5kndglp?")
In ./guix/build/pull.scm:
181:8 1 (build-guix _ _ #:system _ #:storedir _ #:localstatedir ?)
In ice-9/threads.scm:
289:22 0 (loop _)
ice-9/threads.scm:289:22: In procedure loop:
ice-9/threads.scm:289:22: Wrong type (expecting mutex): (3556 . #
Some deprecated features have been used. Set the environment
variable GUILE_WARN_DEPRECATED to "detailed" and rerun the
program to get more information. Set it to "no" to suppress
this message.
builder for `/gnu/store/xxy7l4jfjx6n62anfqlw4gbmafypqgrs-guix-latest.drv' failed with exit code 1
guix pull: error: build failed: build of `/gnu/store/xxy7l4jfjx6n62anfqlw4gbmafypqgrs-guix-latest.drv' failed
signature.asc
(application/pgp-signature, inline)]
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Sat, 24 Jun 2017 16:56:01 GMT)
Full text
and
rfc822 format
available.
Message #8
received at 27476
full text
mbox
):
From:
Leo Famulari
To:
27476
Subject:
Re: Wrong type (expecting mutex)
Date:
Sat, 24 Jun 2017 12:55:08 -0400
Message part 1
(text/plain, inline)]
On Sat, Jun 24, 2017 at 12:32:12PM -0400, Leo Famulari wrote:
> compiling... 99.2% of 606 filesBacktrace:
> 2 (primitive-load "/gnu/store/v9a9cqzh41qg4sixl2mk5kndglp?")
> In ./guix/build/pull.scm:
> 181:8 1 (build-guix _ _ #:system _ #:storedir _ #:localstatedir ?)
> In ice-9/threads.scm:
> 289:22 0 (loop _)
> ice-9/threads.scm:289:22: In procedure loop:
> ice-9/threads.scm:289:22: Wrong type (expecting mutex): (3556 . #
It didn't happen when I re-ran the command.
signature.asc
(application/pgp-signature, inline)]
Severity set to 'important' from 'normal'
Request was from
ludo
to
control
(Sat, 26 Aug 2017 14:14:01 GMT)
Full text
and
rfc822 format
available.
Merged
27476
27652
Request was from
ludo
to
control
(Sat, 26 Aug 2017 14:15:02 GMT)
Full text
and
rfc822 format
available.
Changed bug title to 'Compilation with 'guix pull' crashes non-deterministically on many-core machines' from 'Wrong type (expecting mutex)'
Request was from
ludo
to
control
(Sat, 26 Aug 2017 14:19:01 GMT)
Full text
and
rfc822 format
available.
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Fri, 22 Sep 2017 14:12:01 GMT)
Full text
and
rfc822 format
available.
Message #17
received at 27476
full text
mbox
):
From:
ludo
To:
Ricardo Wurmus
Cc:
help-guix
Subject:
Re: guix pull fails on powerful server
Date:
Fri, 22 Sep 2017 16:10:57 +0200
Hi Ricardo,
Ricardo Wurmus
> The following derivation will be built:
> /gnu/store/yvyfkns3w3vm7ynwbr7mvxcmin4gd2a0-guix-latest.drv
> copying and compiling to '/gnu/store/7m52dkr98nhwgpsx20mmpwyw2yzj58d3-guix-latest' with Guile 2.2.2...
> loading... 25.4% of 629 filesrandom seed for tests: 1506066913
> loading... 99.8% of 629 files
> compiling... 69.2% of 629 filesice-9/threads.scm:289:22: In procedure loop:
> ice-9/threads.scm:289:22: Syntax error:
> guix/scripts.scm:130:2: >>=: >>= (bind) used outside of 'with-monad' in form (>>= (apply set-build-options* #:use-substitutes
> ptions)) (lambda (unused-value) (mbegin %store-monad (mlet %store-monad ((derivation (origin->derivation (package-source pack
> tutes? use-substitutes? #:dry-run? dry-run?) (return (show-derivation-outputs derivation)))))))
This was reported at <
>, and I suspect a
thread-safety issue. However, syntax parameters are purely functional
AFAICS, so I fail to see why multithreading could be a problem.
Andy, any idea what could be causing this?
Thanks,
Ludo’.
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Mon, 25 Sep 2017 07:28:01 GMT)
Full text
and
rfc822 format
available.
Message #20
received at 27476
full text
mbox
):
From:
Andy Wingo
To:
ludo
Cc:
Ricardo Wurmus
27476
Subject:
Re: guix pull fails on powerful server
Date:
Mon, 25 Sep 2017 09:27:45 +0200
On Fri 22 Sep 2017 16:10, ludo
> Hi Ricardo,
> Ricardo Wurmus
>> The following derivation will be built:
>> /gnu/store/yvyfkns3w3vm7ynwbr7mvxcmin4gd2a0-guix-latest.drv
>> copying and compiling to '/gnu/store/7m52dkr98nhwgpsx20mmpwyw2yzj58d3-guix-latest' with Guile 2.2.2...
>> loading... 25.4% of 629 filesrandom seed for tests: 1506066913
>> loading... 99.8% of 629 files
>> compiling... 69.2% of 629 filesice-9/threads.scm:289:22: In procedure loop:
>> ice-9/threads.scm:289:22: Syntax error:
>> guix/scripts.scm:130:2: >>=: >>= (bind) used outside of 'with-monad' in form (>>= (apply set-build-options* #:use-substitutes
>> ptions)) (lambda (unused-value) (mbegin %store-monad (mlet %store-monad ((derivation (origin->derivation (package-source pack
>> tutes? use-substitutes? #:dry-run? dry-run?) (return (show-derivation-outputs derivation)))))))
> This was reported at <
>, and I suspect a
> thread-safety issue. However, syntax parameters are purely functional
> AFAICS, so I fail to see why multithreading could be a problem.
> Andy, any idea what could be causing this?
I have heard of but not seen a number of similar bugs: errors that
"can't happen" but which appear under multiple threads. I don't know
what underlying pattern is. Has anyone found a test case that reliably
reproduces?
Andy
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Mon, 25 Sep 2017 08:44:02 GMT)
Full text
and
rfc822 format
available.
Message #23
received at 27476
full text
mbox
):
From:
Clément Lassieur
To:
Ricardo Wurmus
Cc:
Andy Wingo
Ludovic Courtès
help-guix
Subject:
Re: guix pull fails on powerful server
Date:
Mon, 25 Sep 2017 10:43:16 +0200
I got it too for the first time yesterday on my 128G RAM and 32 CPU
cores server:
--8<---------------cut here---------------start------------->8---
substitute: updating list of substitutes from [...]
Updating from Git repository at '
Building from Git commit 3140844e33254316348135b03762eaeb04764544...
substitute: updating list of substitutes from [...]
The following derivation will be built:
/gnu/store/7143x1dd2r5kch8dldyylz1ljhp3nird-guix-latest.drv
copying and compiling to '/gnu/store/8a42yc4yxslrr1hf7wk5x5mddbs76yqm-guix-latest' with Guile 2.2.2...
loading... 25.3% of 632 filesrandom seed for tests: 1506279202
loading... 99.8% of 632 files
compiling... 94.8% of 632 filesice-9/threads.scm:289:22: In procedure loop:
ice-9/threads.scm:289:22: Syntax error:
guix/gexp.scm:530:8: return: return used outside of 'with-monad' in form (return output)
Some deprecated features have been used. Set the environment
variable GUILE_WARN_DEPRECATED to "detailed" and rerun the
program to get more information. Set it to "no" to suppress
this message.
builder for `/gnu/store/7143x1dd2r5kch8dldyylz1ljhp3nird-guix-latest.drv' failed with exit code 1
guix pull: error: build failed: build of `/gnu/store/7143x1dd2r5kch8dldyylz1ljhp3nird-guix-latest.drv' failed
--8<---------------cut here---------------end--------------->8---
And then, today, again:
--8<---------------cut here---------------start------------->8---
substitute: updating list of substitutes from [...]
Updating from Git repository at '
Building from Git commit 66660960ba75233ae5b6c539f43d97d06d64e9ad...
substitute: updating list of substitutes from [...]
The following derivation will be built:
/gnu/store/dmv64icdan1fqrl00czgwx1an923fzda-guix-latest.drv
copying and compiling to '/gnu/store/slqcrr5gwhi1zqv21wjp0l973zs3dywc-guix-latest' with Guile 2.2.2...
loading... 25.3% of 632 filesrandom seed for tests: 1506327995
loading... 99.8% of 632 files
compiling... 94.8% of 632 filesice-9/threads.scm:289:22: In procedure loop:
ice-9/threads.scm:289:22: Syntax error:
guix/gexp.scm:539:10: return: return used outside of 'with-monad' in form (return (derivation->output-path drv))
Some deprecated features have been used. Set the environment
variable GUILE_WARN_DEPRECATED to "detailed" and rerun the
program to get more information. Set it to "no" to suppress
this message.
builder for `/gnu/store/dmv64icdan1fqrl00czgwx1an923fzda-guix-latest.drv' failed with exit code 1
guix pull: error: build failed: build of `/gnu/store/dmv64icdan1fqrl00czgwx1an923fzda-guix-latest.drv' failed
--8<---------------cut here---------------end--------------->8---
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Mon, 25 Sep 2017 13:04:02 GMT)
Full text
and
rfc822 format
available.
Message #26
received at 27476
full text
mbox
):
From:
ludo
To:
Andy Wingo
Cc:
Ricardo Wurmus
27476
Subject:
Re: guix pull fails on powerful server
Date:
Mon, 25 Sep 2017 15:03:27 +0200
Hi,
Andy Wingo
> On Fri 22 Sep 2017 16:10, ludo
>> Hi Ricardo,
>>
>> Ricardo Wurmus
>>
>>> The following derivation will be built:
>>> /gnu/store/yvyfkns3w3vm7ynwbr7mvxcmin4gd2a0-guix-latest.drv
>>> copying and compiling to '/gnu/store/7m52dkr98nhwgpsx20mmpwyw2yzj58d3-guix-latest' with Guile 2.2.2...
>>> loading... 25.4% of 629 filesrandom seed for tests: 1506066913
>>> loading... 99.8% of 629 files
>>> compiling... 69.2% of 629 filesice-9/threads.scm:289:22: In procedure loop:
>>> ice-9/threads.scm:289:22: Syntax error:
>>> guix/scripts.scm:130:2: >>=: >>= (bind) used outside of 'with-monad' in form (>>= (apply set-build-options* #:use-substitutes
>>> ptions)) (lambda (unused-value) (mbegin %store-monad (mlet %store-monad ((derivation (origin->derivation (package-source pack
>>> tutes? use-substitutes? #:dry-run? dry-run?) (return (show-derivation-outputs derivation)))))))
>>
>> This was reported at <
>, and I suspect a
>> thread-safety issue. However, syntax parameters are purely functional
>> AFAICS, so I fail to see why multithreading could be a problem.
>>
>> Andy, any idea what could be causing this?
> I have heard of but not seen a number of similar bugs: errors that
> "can't happen" but which appear under multiple threads. I don't know
> what underlying pattern is. Has anyone found a test case that reliably
> reproduces?
With this program:
--8<---------------cut here---------------start------------->8---
(use-modules (ice-9 threads)
(srfi srfi-1))
(define-syntax-parameter foo
(identifier-syntax +))
(define threads
(unfold (lambda (x) (> x 100))
(lambda (x)
(call-with-new-thread
(lambda ()
(while #t
(macroexpand
'(syntax-parameterize ((foo (identifier-syntax -)))
(foo y z)))))))
1+
0))
(for-each join-thread threads)
--8<---------------cut here---------------end--------------->8---
I managed to get a segfault:
--8<---------------cut here---------------start------------->8---
$ guile syntax-parms.scm
;;; note: source file /home/ludo/src/guix/syntax-parms.scm
;;; newer than compiled /home/ludo/.cache/guile/ccache/2.2-LE-8-3.A/home/ludo/src/guix/syntax-parms.scm.go
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;; or pass the --no-auto-compile argument to disable.
;;; compiling /home/ludo/src/guix/syntax-parms.scm
;;; compiled /home/ludo/.cache/guile/ccache/2.2-LE-8-3.A/home/ludo/src/guix/syntax-parms.scm.go
In /home/ludo/src/guix/syntax-parms.scm:
13:17 13 (_)
In ice-9/psyntax.scm:
1233:22 12 (expand-top-sequence (#(ribcage #(x) #((m-1dff1b83541ce327-7f97c #)) #) # …) …)
In ice-9/boot-9.scm:
230:17 11 (map1 (#
In ice-9/psyntax.scm:
2053:19 10 (_ _ #() (foo y z) ())
In ice-9/boot-9.scm:
230:17 9 (map1 #())
In ice-9/psyntax.scm:
1408:12 8 (_ _)
1788:11 7 (lp (1) (11 0 . 0))
1678:45 6 (parse _ _ _ _ _ _ _)
In ice-9/boot-9.scm:
230:17 5 (map1 ((tmp-1dff1b83541ce327-7f98b 0 . 0)))
In ice-9/psyntax.scm:
2701:67 4 Adres-eraro
--8<---------------cut here---------------end--------------->8---
… but then I failed to reproduce it again (that was on my 4-thread
laptop).
Ludo’.
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Mon, 25 Sep 2017 14:04:02 GMT)
Full text
and
rfc822 format
available.
Message #29
received at 27476
full text
mbox
):
From:
Ricardo Wurmus
To:
Ludovic Courtès
Cc:
Andy Wingo
Subject:
Re: guix pull fails on powerful server
Date:
Mon, 25 Sep 2017 16:02:36 +0200
Ludovic Courtès
> With this program:
> --8<---------------cut here---------------start------------->8---
> (use-modules (ice-9 threads)
> (srfi srfi-1))
> (define-syntax-parameter foo
> (identifier-syntax +))
> (define threads
> (unfold (lambda (x) (> x 100))
> (lambda (x)
> (call-with-new-thread
> (lambda ()
> (while #t
> (macroexpand
> '(syntax-parameterize ((foo (identifier-syntax -)))
> (foo y z)))))))
> 1+
> 0))
> (for-each join-thread threads)
> --8<---------------cut here---------------end--------------->8---
I have tried this programme on my 16 core 32G workstation, and on the
192 core 1.5T server, but could not get it to segfault.
--
Ricardo
GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Sat, 30 Sep 2017 08:01:01 GMT)
Full text
and
rfc822 format
available.
Message #32
received at 27476
full text
mbox
):
From:
Ricardo Wurmus
To:
Ludovic Courtès
Cc:
Andy Wingo
Subject:
Re: guix pull fails on powerful server
Date:
Sat, 30 Sep 2017 09:59:03 +0200
I’ve tried “guix pull” on the same server again, this time limiting CPUs
with “taskset -c 0 guix pull”:
--8<---------------cut here---------------start------------->8---
$ taskset -c 0 guix pull
guile: warning: failed to install locale
warning: failed to install locale: Invalid argument
substitute: guile: warning: failed to install locale
substitute: warning: failed to install locale: Invalid argument
Starting download of /tmp/guix-file.QleIQR
From
....tar.gz 1.6MiB/s 00:09 | 13.6MiB transferred
unpacking '/gnu/store/g5246hzsj9vv1fmigdd7fh0060fybnbd-guix-latest.tar.gz'...
The following derivation will be built:
/gnu/store/z5bhk17nxmdhvj0g4cy038p25mzh1gys-guix-latest.drv
copying and compiling to '/gnu/store/s3s7xlqa10mvf8v0ypxz8gzw3lcf1x5z-guix-latest' with Guile 2.2.2...
loading... 25.7% of 635 filesrandom seed for tests: 1506720257
loading... 99.8% of 635 files
compiling... 69.1% of 635 filesice-9/threads.scm:289:22: In procedure loop:
ice-9/threads.scm:289:22: Syntax error:
guix/scripts/graph.scm:103:10: return: return used outside of 'with-monad' in form (return (package-node-edges a))
Some deprecated features have been used. Set the environment
variable GUILE_WARN_DEPRECATED to "detailed" and rerun the
program to get more information. Set it to "no" to suppress
this message.
builder for `/gnu/store/z5bhk17nxmdhvj0g4cy038p25mzh1gys-guix-latest.drv' failed with exit code 1
guix pull: error: build failed: build of `/gnu/store/z5bhk17nxmdhvj0g4cy038p25mzh1gys-guix-latest.drv' failed
--8<---------------cut here---------------end--------------->8---
After limiting memory with “ulimit -Sv 5000000”:
--8<---------------cut here---------------start------------->8---
ice-9/threads.scm:289:22: In procedure loop:
ice-9/threads.scm:289:22: Syntax error:
guix/scripts/pull.scm:192:8: >>=: >>= (bind) used outside of 'with-monad' in form (>>= (indirect-root-added latest) (lambda (done) (mlet* %store-monad () (if (and (file-exists? latest) (string=? (readlink latest) source-dir)) (begin (display (G_ "Guix already up to date\n")) (return #t)) (begin (switch-symlinks latest source-dir) (format #t (G_ "updated ~a successfully deployed under `~a'~%") %guix-package-name latest) (return #t))))))
Some deprecated features have been used. Set the environment
variable GUILE_WARN_DEPRECATED to "detailed" and rerun the
program to get more information. Set it to "no" to suppress
this message.
builder for `/gnu/store/afji58647yzz7cr9dvlj87sd3ad36lbk-guix-latest.drv' failed with exit code 1
guix pull: error: build failed: build of `/gnu/store/afji58647yzz7cr9dvlj87sd3ad36lbk-guix-latest.drv' failed
--8<---------------cut here---------------end--------------->8---
It always crashes at around 69%.
Is there another work-around I could try on this machine?
--
Ricardo
GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Tue, 03 Oct 2017 20:30:02 GMT)
Full text
and
rfc822 format
available.
Message #35
received at 27476
full text
mbox
):
From:
Marius Bakke
To:
Ricardo Wurmus
Cc:
Andy Wingo
Subject:
Re: guix pull fails on powerful server
Date:
Tue, 03 Oct 2017 22:29:00 +0200
Message part 1
(text/plain, inline)]
Ricardo Wurmus
> Is there another work-around I could try on this machine?
Using Guile 2.0 worked for me:
guix package -r guix guile-git -i guile2.0-guix guile2.0-git
signature.asc
(application/pgp-signature, inline)]
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Wed, 04 Oct 2017 13:18:01 GMT)
Full text
and
rfc822 format
available.
Message #38
received at 27476
full text
mbox
):
From:
Ricardo Wurmus
To:
Marius Bakke
Cc:
Andy Wingo
Ludovic Courtès
27476
Subject:
Re: guix pull fails on powerful server
Date:
Wed, 04 Oct 2017 15:15:58 +0200
Marius Bakke
> Ricardo Wurmus
>> Is there another work-around I could try on this machine?
> Using Guile 2.0 worked for me:
> guix package -r guix guile-git -i guile2.0-guix guile2.0-git
Unfortunately, this didn’t work for me. I tried this:
guix package -i guile2.0-guix --with-source=/path/to/guix/checkout
This printed a lot of repetitions of “warning: unknown warning type
`macro-use-before-definition'” and eventually failed with this
backtrace:
--8<---------------cut here---------------start------------->8---
Backtrace:
In ice-9/boot-9.scm:
160: 9 [catch #t #
In unknown file:
?: 8 [apply-smob/1 #
In ice-9/boot-9.scm:
66: 7 [call-with-prompt prompt0 ...]
In ice-9/eval.scm:
432: 6 [eval # #]
In ice-9/boot-9.scm:
2412: 5 [save-module-excursion #
4091: 4 [#
1734: 3 [%start-stack load-stack #
1739: 2 [#
In unknown file:
?: 1 [primitive-load "/tmp/guix-build-guile2.0-guix-0.13.0-4.f1ddfe4.drv-0/source/./build-aux/compile-all.scm"]
In ice-9/threads.scm:
99: 0 [loop (("guix/base16.scm" "guix/base32.scm" "guix/base64.scm" ...))]
ice-9/threads.scm:99:22: In procedure loop:
ice-9/threads.scm:99:22: In procedure fport_write: Bad address
make[2]: *** [Makefile:5252: make-go] Error 1
make[2]: Leaving directory '/tmp/guix-build-guile2.0-guix-0.13.0-4.f1ddfe4.drv-0/source'
make[1]: *** [Makefile:4383: all-recursive] Error 1
make[1]: Leaving directory '/tmp/guix-build-guile2.0-guix-0.13.0-4.f1ddfe4.drv-0/source'
make: *** [Makefile:2973: all] Error 2
phase `build' failed after 149.5 seconds
builder for `/gnu/store/aqz4d2bbihvdmxqb6rlb71c853jb4dp3-guile2.0-guix-0.13.0-4.f1ddfe4.drv' failed with exit code 1
guix package: error: build failed: build of `/gnu/store/aqz4d2bbihvdmxqb6rlb71c853jb4dp3-guile2.0-guix-0.13.0-4.f1ddfe4.drv
--8<---------------cut here---------------end--------------->8---
--
Ricardo
GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Wed, 04 Oct 2017 15:10:01 GMT)
Full text
and
rfc822 format
available.
Message #41
received at 27476
full text
mbox
):
From:
Clément Lassieur
To:
Ricardo Wurmus
Cc:
Andy Wingo
Ludovic Courtès
27476
Subject:
Re: guix pull fails on powerful server
Date:
Wed, 04 Oct 2017 17:09:41 +0200
Ricardo Wurmus
> Is there another work-around I could try on this machine?
My workaround was to build Guix from sources. But I'm sure you thought
about it.
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Wed, 04 Oct 2017 16:19:01 GMT)
Full text
and
rfc822 format
available.
Message #44
received at 27476
full text
mbox
):
From:
Ricardo Wurmus
To:
Clément Lassieur
Cc:
Andy Wingo
Ludovic Courtès
27476
Subject:
Re: guix pull fails on powerful server
Date:
Wed, 04 Oct 2017 18:17:57 +0200
Clément Lassieur
> Ricardo Wurmus
>> Is there another work-around I could try on this machine?
> My workaround was to build Guix from sources. But I'm sure you thought
> about it.
Yes, that works, but I was looking for something that a user can do who
wouldn’t like to fiddle with git.
--
Ricardo
GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Sat, 07 Oct 2017 15:12:02 GMT)
Full text
and
rfc822 format
available.
Message #47
received at 27476
full text
mbox
):
From:
ludo
To:
Ricardo Wurmus
Cc:
Andy Wingo
Subject:
Re: guix pull fails on powerful server
Date:
Sat, 07 Oct 2017 17:11:42 +0200
Ricardo Wurmus
> I’ve tried “guix pull” on the same server again, this time limiting CPUs
> with “taskset -c 0 guix pull”:
As a stopgap, commit aba219af0fed6a349af930f19c913fb87e6a69dd ensures
that ‘--cores’ is honored. So if you run “guix pull --cores=1”, it will
build things sequentially.
Now, to take advantage of that, you first need to update to the current
Guix…
Ludo’.
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Tue, 10 Oct 2017 07:19:01 GMT)
Full text
and
rfc822 format
available.
Message #50
received at 27476
full text
mbox
):
From:
Ricardo Wurmus
To:
Ludovic Courtès
Cc:
Andy Wingo
Subject:
Re: guix pull fails on powerful server
Date:
Tue, 10 Oct 2017 09:17:17 +0200
Ludovic Courtès
> Ricardo Wurmus
>> I’ve tried “guix pull” on the same server again, this time limiting CPUs
>> with “taskset -c 0 guix pull”:
> As a stopgap, commit aba219af0fed6a349af930f19c913fb87e6a69dd ensures
> that ‘--cores’ is honored. So if you run “guix pull --cores=1”, it will
> build things sequentially.
> Now, to take advantage of that, you first need to update to the current
> Guix…
Thank you, this worked!
On that server I built Guix from source and then let the users pull with
“--cores=1” to update their own Guix.
--
Ricardo
GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Tue, 10 Oct 2017 11:33:01 GMT)
Full text
and
rfc822 format
available.
Message #53
received at 27476
full text
mbox
):
From:
ludo
To:
Ricardo Wurmus
Cc:
Andy Wingo
Subject:
Re: guix pull fails on powerful server
Date:
Tue, 10 Oct 2017 13:32:30 +0200
Ricardo Wurmus
> Ludovic Courtès
>> Ricardo Wurmus
>>
>>> I’ve tried “guix pull” on the same server again, this time limiting CPUs
>>> with “taskset -c 0 guix pull”:
>>
>> As a stopgap, commit aba219af0fed6a349af930f19c913fb87e6a69dd ensures
>> that ‘--cores’ is honored. So if you run “guix pull --cores=1”, it will
>> build things sequentially.
>>
>> Now, to take advantage of that, you first need to update to the current
>> Guix…
> Thank you, this worked!
> On that server I built Guix from source and then let the users pull with
> “--cores=1” to update their own Guix.
You could also run guix-daemon with --cores=4 or similar, so that it
uses 4 cores by default (few package builds scale beyond that anyway),
and then maybe --max-jobs=4 so you don’t waste the other cores. ;-)
Ludo’.
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Thu, 12 Oct 2017 13:38:02 GMT)
Full text
and
rfc822 format
available.
Message #56
received at 27476
full text
mbox
):
From:
ludo
To:
Ricardo Wurmus
Cc:
Andy Wingo
Subject:
Re: bug#27476: guix pull fails on powerful server
Date:
Thu, 12 Oct 2017 15:37:12 +0200
Message part 1
(text/plain, inline)]
Hi!
Ricardo Wurmus
> The following derivation will be built:
> /gnu/store/z5bhk17nxmdhvj0g4cy038p25mzh1gys-guix-latest.drv
> copying and compiling to '/gnu/store/s3s7xlqa10mvf8v0ypxz8gzw3lcf1x5z-guix-latest' with Guile 2.2.2...
> loading... 25.7% of 635 filesrandom seed for tests: 1506720257
> loading... 99.8% of 635 files
> compiling... 69.1% of 635 filesice-9/threads.scm:289:22: In procedure loop:
> ice-9/threads.scm:289:22: Syntax error:
> guix/scripts/graph.scm:103:10: return: return used outside of 'with-monad' in form (return (package-node-edges a))
The program below crashes with completely surreal backtraces in less
than a minute on my 4-thread laptop:
--8<---------------cut here---------------start------------->8---
(use-modules (ice-9 threads)
(srfi srfi-1)
(guix monads)
(guix store))
(define threads
(unfold (lambda (x) (> x 100))
(lambda (x)
(call-with-new-thread
(lambda ()
(define monad
(symbol-append 'foo-monad
(string->symbol (number->string x))))
(while #t
(macroexpand
`(begin
(define-monad ,monad
(bind +)
(return -))
(with-monad ,monad
(return 3))
(mapm ,monad + '(1 2 3))))))))
1+
0))
(for-each join-thread threads)
--8<---------------cut here---------------end--------------->8---
Can you check if that also happens on your many-core machine?
The patch below seems to fix the problem: (guix monads) has shared state
(hash tables) used both at expansion-time and run-time, and it wasn’t
protected.
My hypothesis is that this was causing random memory corruption. The
weird thing, though, is that the errors we were getting were not so
random. Also, the load phase of ‘guix pull’ is sequential.
Could you test it and report back?
Thanks,
Ludo’.
Message part 2
(text/x-patch, inline)]
diff --git a/guix/monads.scm b/guix/monads.scm
index 6ae616aca..c9c5da3bb 100644
--- a/guix/monads.scm
+++ b/guix/monads.scm
@@ -20,6 +20,7 @@
#:use-module ((system syntax)
#:select (syntax-local-binding))
#:use-module (ice-9 match)
+ #:use-module (ice-9 threads)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-9)
#:use-module (srfi srfi-26)
@@ -117,6 +118,7 @@
;; the syntax object of the parameter over which it is templated, and (2)
;; the syntax of its body.
(define-once %templates (make-hash-table))
+ (define-once %template-lock (make-mutex))
(define (register-template! name param body)
(hash-set! %templates name (cons param body)))
@@ -139,8 +141,9 @@ template instances."
(syntax-source s))
(define current-info-port
- ;; Port for debugging info.
- (const (%make-void-port "w")))
+ ;; Port for debugging info. Return a fresh port at each call to make
+ ;; sure we're thread-safe.
+ (lambda () (%make-void-port "w")))
(define location-string
(format #f "~a:~a:~a"
@@ -204,12 +207,14 @@ template instances."
;; Search for an instance of template NAME for this ACTUAL parameter.
;; On success, expand to the identifier of the instance; otherwise
;; expand to #f.
- (any (matching-instance? #'name #'actual) %template-instances))
+ (with-mutex %template-lock
+ (any (matching-instance? #'name #'actual) %template-instances)))
((_ exists? name actual)
;; Likewise, but return a Boolean.
(let ((result (->bool
- (any (matching-instance? #'name #'actual)
- %template-instances))))
+ (with-mutex %template-lock
+ (any (matching-instance? #'name #'actual)
+ %template-instances)))))
(unless result
(format (current-warning-port)
"~a: warning: no specialization of template '~a' for '~a'~%"
@@ -220,8 +225,9 @@ template instances."
;; Expand to the definitions of all the existing templates
;; specialized for ACTUAL.
#`(begin
- #,@(hash-map->list (cut instance-definition <> <> #'actual)
- %templates))))))
+ #,@(with-mutex %template-lock
+ (hash-map->list (cut instance-definition <> <> #'actual)
+ %templates)))))))
(define-syntax define-template
(lambda (s)
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Fri, 13 Oct 2017 20:31:02 GMT)
Full text
and
rfc822 format
available.
Message #59
received at 27476
full text
mbox
):
From:
Ricardo Wurmus
To:
Ludovic Courtès
Cc:
Andy Wingo
Subject:
Re: bug#27476: guix pull fails on powerful server
Date:
Fri, 13 Oct 2017 22:29:09 +0200
Hi Ludo,
> Ricardo Wurmus
>> The following derivation will be built:
>> /gnu/store/z5bhk17nxmdhvj0g4cy038p25mzh1gys-guix-latest.drv
>> copying and compiling to '/gnu/store/s3s7xlqa10mvf8v0ypxz8gzw3lcf1x5z-guix-latest' with Guile 2.2.2...
>> loading... 25.7% of 635 filesrandom seed for tests: 1506720257
>> loading... 99.8% of 635 files
>> compiling... 69.1% of 635 filesice-9/threads.scm:289:22: In procedure loop:
>> ice-9/threads.scm:289:22: Syntax error:
>> guix/scripts/graph.scm:103:10: return: return used outside of 'with-monad' in form (return (package-node-edges a))
> The program below crashes with completely surreal backtraces in less
> than a minute on my 4-thread laptop:
> --8<---------------cut here---------------start------------->8---
> (use-modules (ice-9 threads)
> (srfi srfi-1)
> (guix monads)
> (guix store))
> (define threads
> (unfold (lambda (x) (> x 100))
> (lambda (x)
> (call-with-new-thread
> (lambda ()
> (define monad
> (symbol-append 'foo-monad
> (string->symbol (number->string x))))
> (while #t
> (macroexpand
> `(begin
> (define-monad ,monad
> (bind +)
> (return -))
> (with-monad ,monad
> (return 3))
> (mapm ,monad + '(1 2 3))))))))
> 1+
> 0))
> (for-each join-thread threads)
> --8<---------------cut here---------------end--------------->8---
> Can you check if that also happens on your many-core machine?
It does not crash. I left it running for more than an hour (without
compiling) and it printed things like this:
--8<---------------cut here---------------start------------->8---
GC Warning: Repeated allocation of very large block (appr. size 57528320):
May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 57528320):
May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 57528320):
May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 57528320):
May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 14385152):
May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 14385152):
May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 57528320):
May lead to memory leak and poor performance
GC Warning: Repeated allocation of very large block (appr. size 28766208):
May lead to memory leak and poor performance
--8<---------------cut here---------------end--------------->8---
That’s on the machine with 1.5T RAM and 192 cores. Then I ran it again
for 10 minutes after compiling it. It did not crash.
> The patch below seems to fix the problem: (guix monads) has shared state
> (hash tables) used both at expansion-time and run-time, and it wasn’t
> protected.
> My hypothesis is that this was causing random memory corruption. The
> weird thing, though, is that the errors we were getting were not so
> random. Also, the load phase of ‘guix pull’ is sequential.
> Could you test it and report back?
I’m trying the patch right now with “guix pull”.
--
Ricardo
GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Fri, 13 Oct 2017 21:06:01 GMT)
Full text
and
rfc822 format
available.
Message #62
received at 27476
full text
mbox
):
From:
Ricardo Wurmus
To:
Ludovic Courtès
Cc:
Andy Wingo
Subject:
Re: bug#27476: guix pull fails on powerful server
Date:
Fri, 13 Oct 2017 23:04:31 +0200
Hi Ludo,
> The patch below seems to fix the problem: (guix monads) has shared state
> (hash tables) used both at expansion-time and run-time, and it wasn’t
> protected.
> My hypothesis is that this was causing random memory corruption. The
> weird thing, though, is that the errors we were getting were not so
> random. Also, the load phase of ‘guix pull’ is sequential.
> Could you test it and report back?
This doesn’t seem to be enough to fix the problem. I patched ~/guix and
ran “guix pull --url=$PWD” from ~/guix:
--8<---------------cut here---------------start------------->8---
[rwurmus
guile: warning: failed to install locale
warning: failed to install locale: Invalid argument
Updating from Git repository at '/home/rwurmus/guix'...
Building from Git commit d24c69d86670bfad0c6bb147162c918e9fcdccc2...
substitute: guile: warning: failed to install locale
substitute: warning: failed to install locale: Invalid argument
guix pull: warning: failed to load '(bimsb packages bioinformatics-nonfree)':
ERROR: no code for module (gnu packages zip)
guix pull: warning: failed to load '(bimsb packages staging)':
ERROR: no code for module (gnu packages zip)
substitute: updating list of substitutes from '
. 100.0%
The following derivation will be built:
/gnu/store/y54b92jj44li36743fgxzy0iagi6gb9n-guix-latest.drv
copying and compiling to '/gnu/store/p5zlw7fas26bickkqz4d68g8bxnjr14z-guix-latest' with Guile 2.2.2...
loading... 25.8% of 640 filesrandom seed for tests: 1507927861
loading... 99.8% of 640 files
compiling... 18.6% of 640 filesIn thread:
ERROR: In procedure return: return used outside of 'with-monad'Error while printing exception.
compiling... 70.0% of 640 files
--8<---------------cut here---------------end--------------->8---
The higher the percentage of completion, the slower this all gets. It
hasn’t actually finished yet, even though it has been running for over
13 minutes.
--
Ricardo
GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Fri, 13 Oct 2017 21:12:01 GMT)
Full text
and
rfc822 format
available.
Message #65
received at 27476
full text
mbox
):
From:
Ricardo Wurmus
To:
Ludovic Courtès
Cc:
Andy Wingo
Subject:
Re: bug#27476: guix pull fails on powerful server
Date:
Fri, 13 Oct 2017 23:10:29 +0200
I tried it again after unsetting GUIX_PACKAGE_PATH, but the results are
just as bad:
--8<---------------cut here---------------start------------->8---
[rwurmus
[rwurmus
guile: warning: failed to install locale
warning: failed to install locale: Invalid argument
Updating from Git repository at '/home/rwurmus/guix'...
Building from Git commit d24c69d86670bfad0c6bb147162c918e9fcdccc2...
substitute: guile: warning: failed to install locale
substitute: warning: failed to install locale: Invalid argument
substitute: updating list of substitutes from '
. 100.0%
The following derivation will be built:
/gnu/store/q5sh4c1mfk396kixqdq8j0wdfwin4dsx-guix-latest.drv
copying and compiling to '/gnu/store/jzq053lg77shnysmhj4i2f2bngz2rr5b-guix-latest' with Guile 2.2.2...
loading... 25.8% of 640 filesrandom seed for tests: 1507928738
loading... 99.8% of 640 files
compiling... 14.4% of 640 filesIn thread:
ERROR: In procedure >>=: >>= (bind) used outside of 'with-monad'Error while printing exception.
compiling... 14.5% of 640 filesIn thread:
ERROR: In procedure return: return used outside of 'with-monad'Error while printing exception.
compiling... 17.3% of 640 filesIn thread:
ERROR: In procedure return: return used outside of 'with-monad'Error while printing exception.
compiling... 70.0% of 640 filesGC Warning: Repeated allocation of very large block (appr. size 28766208):
May lead to memory leak and poor performance
--8<---------------cut here---------------end--------------->8---
--
Ricardo
GPG: BCA6 89B6 3655 3801 C3C6 2150 197A 5888 235F ACAC
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Tue, 07 Nov 2017 10:58:02 GMT)
Full text
and
rfc822 format
available.
Message #68
received at 27476
full text
mbox
):
From:
ludo
To:
Ricardo Wurmus
Cc:
Andy Wingo
Subject:
Re: bug#27476: guix pull fails on powerful server
Date:
Tue, 07 Nov 2017 11:57:10 +0100
Hi,
Ricardo Wurmus
> After limiting memory with “ulimit -Sv 5000000”:
> ice-9/threads.scm:289:22: In procedure loop:
> ice-9/threads.scm:289:22: Syntax error:
> guix/scripts/pull.scm:192:8: >>=: >>= (bind) used outside of 'with-monad' in form (>>= (indirect-root-added latest) (lambda (done) (mlet* %store-monad () (if (and (file-exists? latest) (string=? (readlink latest) source-dir)) (begin (display (G_ "Guix already up to date\n")) (return #t)) (begin (switch-symlinks latest source-dir) (format #t (G_ "updated ~a successfully deployed under `~a'~%") %guix-package-name latest) (return #t))))))
> Some deprecated features have been used. Set the environment
> variable GUILE_WARN_DEPRECATED to "detailed" and rerun the
> program to get more information. Set it to "no" to suppress
> this message.
> builder for `/gnu/store/afji58647yzz7cr9dvlj87sd3ad36lbk-guix-latest.drv' failed with exit code 1
> guix pull: error: build failed: build of `/gnu/store/afji58647yzz7cr9dvlj87sd3ad36lbk-guix-latest.drv' failed
> It always crashes at around 69%.
This gave me an idea. With this program:
--8<---------------cut here---------------start------------->8---
(use-modules (ice-9 threads)
(srfi srfi-1)
(guix monads)
(guix store)
(system base compile))
(compile #f) ;load modules
(define threads
(unfold (lambda (x) (> x 100))
(lambda (x)
(call-with-new-thread
(lambda ()
(while #t
(compile
'(begin
(with-monad %store-monad
(>>= foo bar
(return 3)))
(mlet %store-monad ((x y))
(mbegin %store-monad
(return x)
(return y))))
#:env (current-module)
#:from 'scheme
#:to 'tree-il)))))
1+
0))
(for-each join-thread threads)
--8<---------------cut here---------------end--------------->8---
I can reproduce the error:
--8<---------------cut here---------------start------------->8---
$ ulimit -Sv 2000000
$ guile syntax-parms.scm
In ice-9/psyntax.scm:
1678:45 19 (parse _ _ _ _ _ _ _)
In ice-9/boot-9.scm:
230:17 18 (map1 (((("placeholder" placeholder) ((#
In ice-9/psyntax.scm:
1483:23 17 (_ _ _)
In ice-9/boot-9.scm:
230:29 16 (map1 (#
230:17 15 (map1 (#
In ice-9/psyntax.scm:
1788:11 14 (lp ((#
1678:45 13 (parse _ _ _ _ _ _ _)
In ice-9/boot-9.scm:
230:17 12 (map1 (((("placeholder" placeholder) ("l-1dff1b83541ce327-67a3671" lexical . #) ("placeho…" …) …) …)))
In ice-9/psyntax.scm:
1678:45 11 (parse _ _ _ _ _ _ _)
In ice-9/boot-9.scm:
230:17 10 (map1 (((("placeholder" placeholder) ((#
In ice-9/psyntax.scm:
2337:44 9 (expand-let _ _ _ #f (hygiene guile-user) #
1678:45 8 (parse _ _ _ _ _ _ _)
In ice-9/boot-9.scm:
230:17 7 (map1 (((("placeholder" placeholder) ("l-1dff1b83541ce327-67a37b2" lexical . #) ("placeho…" …) …) …)))
In ice-9/psyntax.scm:
1678:45 6 (parse _ _ _ _ _ _ _)
In ice-9/boot-9.scm:
230:17 5 (map1 (((("placeholder" placeholder) ((#
In ice-9/psyntax.scm:
1483:23 4 (_ _ _)
In ice-9/boot-9.scm:
230:17 3 (map1 (#
In ice-9/psyntax.scm:
1406:23 2 (_ _)
1347:32 1 (syntax-type ((#
1558:32 0 (expand-macro #
ice-9/psyntax.scm:1558:32: In procedure expand-macro:
ice-9/psyntax.scm:1558:32: Syntax error:
unknown location: state-return: Wrong number of arguments in form ((%store-monad %return))
In syntax-parms.scm:
15:17 9 (_)
In system/base/compile.scm:
255:6 8 (compile _ #:from _ #:to _ #:env _ #:opts _)
183:32 7 (compile-fold (#
In ice-9/boot-9.scm:
2316:4 6 (save-module-excursion #
In language/scheme/compile-tree-il.scm:
31:15 5 (_)
In ice-9/psyntax.scm:
1233:22 4 (expand-top-sequence ((begin (with-monad %store-monad (>>= foo bar (return 3))) (mlet # ((…)) #))) _ …)
In ice-9/boot-9.scm:
230:17 3 (map1 (#
In ice-9/psyntax.scm:
1611:33 2 (parse (((("placeholder" placeholder) ((#
1347:32 1 (syntax-type (>>= foo bar (return 3)) (("placeholder" placeholder) ((#
1558:32 0 (expand-macro #
ice-9/psyntax.scm:1558:32: In procedure expand-macro:
ice-9/psyntax.scm:1558:32: Syntax error:
unknown location: source expression failed to match any pattern
GC Warning: Failed to expand heap by 28770304 bytes
GC Warning: Failed to expand heap by 28770304 bytes
GC Warning: Failed to expand heap by 14385152 bytes
GC Warning: Out of Memory! Heap size: 919 MiB. Returning NULL!
Warning: Unwind-only `out-of-memory' exception; skipping pre-unwind handler.
--8<---------------cut here---------------end--------------->8---
So it looks like Guile failing badly in ENOMEM conditions.
I can’t reproduce this with current Guile ‘stable-2.2’, following Andy’s
weak-table rewrite¹, so this might have been a weak-table bug showing up
under memory pressure.
With ‘guix pull’ this was more likely to happen on your many-core server
than on my laptop because you have more threads and thus much higher
memory usage.
Ludo’.
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Mon, 30 Apr 2018 21:20:02 GMT)
Full text
and
rfc822 format
available.
Message #71
received at 27476
full text
mbox
):
From:
ludo
To:
Ricardo Wurmus
Cc:
Andy Wingo
Subject:
Re: bug#27476: guix pull fails on powerful server
Date:
Mon, 30 Apr 2018 23:19:42 +0200
ludo
> I can’t reproduce this with current Guile ‘stable-2.2’, following Andy’s
> weak-table rewrite¹, so this might have been a weak-table bug showing up
> under memory pressure.
With Guile 2.2.3 a similar program triggers a crash very quickly:
--8<---------------cut here---------------start------------->8---
$ cat ../guile-debugging/syntax-parms.scm
(use-modules (ice-9 threads)
(srfi srfi-1)
(guix monads)
(guix store)
(system base compile))
(compile #f) ;load modules
(define threads
(unfold (lambda (x) (> x 100))
(lambda (x)
(call-with-new-thread
(lambda ()
(while #t
(compile
'(mlet %store-monad ((x y))
(mbegin %store-monad
(return x)
(return y)))
#:env (current-module)
#:from 'scheme
#:to 'tree-il)))))
1+
0))
(for-each join-thread threads)
$ guile --version
guile (GNU Guile) 2.2.3
Copyright (C) 2017 Free Software Foundation, Inc.
License LGPLv3+: GNU LGPL 3 or later <
>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
$ guile ../guile-debugging/syntax-parms.scm
In ice-9/psyntax.scm:
2338:44 19 (expand-let _ _ _ #f _ #
1679:45 18 (parse _ _ _ _ _ _ _)
In ice-9/boot-9.scm:
222:17 17 (map1 ("-" "1dff1b83541ce327" "-" "2ad70"))
In ice-9/psyntax.scm:
1679:45 16 (parse _ _ _ _ _ _ _)
In ice-9/boot-9.scm:
222:17 15 (map1 ((#(ribcage () () ()) #(ribcage () () ()) #(ribcage #(unused-value) #((m-1dff1b83541ce327-29a7a top)) #("l-1dff1b83541ce327-2bce9")) #(ribcage () () ()) # …)))
In ice-9/psyntax.scm:
1484:23 14 (_ _ _)
In ice-9/boot-9.scm:
222:29 13 (map1 _)
222:17 12 (map1 ("-" "2bd28"))
In ice-9/psyntax.scm:
1789:11 11 (lp _ ())
1679:45 10 (parse _ _ _ _ _ _ _)
In ice-9/boot-9.scm:
222:17 9 (map1 ((expand-1dff1b83541ce327-2bd31) (#
In ice-9/psyntax.scm:
1679:45 8 (parse _ _ _ _ _ _ _)
In ice-9/boot-9.scm:
222:17 7 (map1 ((m-1dff1b83541ce327-2bcf4 top)))
In ice-9/psyntax.scm:
1484:23 6 (_ _ _)
In ice-9/boot-9.scm:
222:17 5 (map1 (#
In ice-9/psyntax.scm:
1407:23 4 (_ _)
1317:39 3 (syntax-type y (shift #(ribcage #(e) #((top)) #("l-680b775fb37a463-1343")) #(ribcage () () ()) #(ribcage #(xx) #((top)) #("l-680b775fb37a463-1340")) #(ribcage …)) # …)
916:15 2 (resolve-identifier y (#
833:21 1 (id-var-name y _ _)
669:4 0 (search y (() #(ribcage () () ()) #(ribcage () () ()) #(ribcage () () ()) #(ribcage () () ())) (top) (hygiene guile-user))
ice-9/psyntax.scm:669:4: In procedure search:
In procedure vector-ref: Wrong type argument in position 1 (expecting vector): ()
In ice-9/boot-9.scm:
222:29 19 (map1 _)
222:17 18 (map1 (#f))
In ice-9/psyntax.scm:
1789:11 17 (lp _ ())
1679:45 16 (parse _ _ _ _ _ _ _)
In ice-9/boot-9.scm:
222:17 15 (map1 (() (m-1dff1b83541ce327-8f24e top)))
In ice-9/psyntax.scm:
1679:45 14 (parse _ _ _ _ _ _ _)
In ice-9/boot-9.scm:
222:17 13 (map1 ((m-1dff1b83541ce327-8fe71 top) shift #(ribcage #(e) #((top)) #("l-680b775fb37a463-1343")) #(ribcage () () ()) #(ribcage #(xx) #((top)) #("l-680b775fb3…")) #))
In ice-9/psyntax.scm:
1409:12 12 (expand-expr _ _ _ (shift #(ribcage #(e) #((top)) #("l-680b775fb37a463-1343")) #(ribcage () () ()) #(ribcage #(xx) #((top)) #("l-680b775fb37a463-1340")) #(# # …)) # …)
2054:19 11 (_ _ _ (m-1dff1b83541ce327-8fe71 top) ())
In ice-9/boot-9.scm:
222:17 10 (map1 (#
In ice-9/psyntax.scm:
1409:12 9 (_ _)
1789:11 8 (lp _ (#(ribcage () () ()) #(ribcage #(x) #((m-1dff1b83541ce327-8f383 top)) #("l-1dff1b83541ce327-8f40b")) #(ribcage () () ()) #(ribcage () () ()) #(ribcage # …) …))
1679:45 7 (parse _ _ _ _ _ _ _)
In ice-9/boot-9.scm:
222:17 6 (map1 (#(ribcage #(x) #((m-1dff1b83541ce327-8f383 top)) #("l-1dff1b83541ce327-8f40b")) #(ribcage () () ()) shift #(ribcage #(monad body) #((top) (top)) #("…" …)) …))
In ice-9/psyntax.scm:
2702:67 5 (_ tmp-1dff1b83541ce327-8fa7a _ _)
2646:111 4 (gen-clause #
2607:69 3 (build-dispatch-call (("8fa7c" . 0)) "1dff1b83541ce327" #
In ice-9/boot-9.scm:
222:17 2 (map1 ("8fa7c"))
In ice-9/psyntax.scm:
2004:10 1 (gen-var _)
In unknown file:
0 (symbol->string "8fa7c")
ERROR: In procedure symbol->string:
In procedure symbol->string: Wrong type argument in position 1 (expecting symbol): "8fa7c"
--8<---------------cut here---------------end--------------->8---
Ludo’.
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Mon, 30 Apr 2018 21:40:01 GMT)
Full text
and
rfc822 format
available.
Message #74
received at 27476
full text
mbox
):
From:
ludo
To:
Ricardo Wurmus
Cc:
Andy Wingo
Subject:
libguile/memoize.c is not thread safe,
so syntax parameter expansion is not thread-safe
Date:
Mon, 30 Apr 2018 23:39:05 +0200
ludo
> With Guile 2.2.3 a similar program triggers a crash very quickly:
Even simpler:
--8<---------------cut here---------------start------------->8---
$ guile ../guile-debugging/syntax-parms.scm
;;; note: source file /home/ludo/src/guix/../guile-debugging/syntax-parms.scm
;;; newer than compiled /home/ludo/.cache/guile/ccache/2.2-LE-8-3.A/home/ludo/src/guile-debugging/syntax-parms.scm.go
;;; note: auto-compilation is enabled, set GUILE_AUTO_COMPILE=0
;;; or pass the --no-auto-compile argument to disable.
;;; compiling /home/ludo/src/guix/../guile-debugging/syntax-parms.scm
;;; compiled /home/ludo/.cache/guile/ccache/2.2-LE-8-3.A/home/ludo/src/guile-debugging/syntax-parms.scm.go
In /home/ludo/src/guix/../guile-debugging/syntax-parms.scm:
37:17 13 (_)
In system/base/compile.scm:
255:6 12 (compile _ #:from _ #:to _ #:env _ #:opts _)
183:32 11 (compile-fold (#f syntax-violation (#
In ice-9/boot-9.scm:
2312:4 10 (save-module-excursion #
In language/scheme/compile-tree-il.scm:
31:15 9 (_)
In ice-9/psyntax.scm:
1234:22 8 (expand-top-sequence (#
In ice-9/boot-9.scm:
222:29 7 (map1 ((10 (3 #((0 . 0)) 2 (1 (11 0 . 0) (7 (3 #() 2 (10 (13 15 5 (guile) list . #f) (5 . #
222:17 6 (map1 (10 (3 #((0 . 0)) 2 (1 (11 0 . 0) (7 (3 #() 2 (10 (13 15 5 (guile) list . #f) (5 . #
In ice-9/psyntax.scm:
2054:19 5 (_ _ (#
In ice-9/boot-9.scm:
222:17 4 (map1 (#
In ice-9/psyntax.scm:
2057:27 3 (_ _)
289:10 2 (eval-local-transformer _ _)
In ice-9/eval.scm:
718:15 1 (primitive-eval _)
In unknown file:
0 (memoize-expression #
ERROR: In procedure memoize-expression:
In procedure vector: Wrong type argument in position 1: #(#
C-c C-c
$ cat ../guile-debugging/syntax-parms.scm
(use-modules (ice-9 threads)
(srfi srfi-1)
(system base compile))
(compile #f) ;load modules
(define-syntax-parameter param
(lambda (s)
(syntax-case s ()
((_ a b) #'(+ a b)))))
(define threads
(unfold (lambda (x) (> x 100))
(lambda (x)
(call-with-new-thread
(lambda ()
(while #t
(compile '(begin
(param 1 2)
(syntax-parameterize ((param (lambda (s)
(syntax-case s ()
((_ a b)
#'(- a b))))))
(param 1 2)))
#:env (current-module)
#:from 'scheme
#:to 'tree-il)))))
1+
0))
(for-each join-thread threads)
--8<---------------cut here---------------end--------------->8---
So the problem, AIUI, is that psyntax evaluates syntax parameters using
‘primitive-eval’ (via ‘eval-local-transformer’), but memoization in
(ice-9 eval) is not thread-safe, hence the random crashes.
Not sure what can be done. Thoughts?
Ludo’.
Changed bug title to 'Multi-threaded compilation of 'syntax-parameterize' forms crashes' from 'Compilation with 'guix pull' crashes non-deterministically on many-core machines'
Request was from
ludo
to
control
(Mon, 30 Apr 2018 21:41:01 GMT)
Full text
and
rfc822 format
available.
Severity set to 'serious' from 'important'
Request was from
ludo
to
control
(Mon, 30 Apr 2018 21:41:02 GMT)
Full text
and
rfc822 format
available.
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Wed, 09 May 2018 08:42:02 GMT)
Full text
and
rfc822 format
available.
Message #81
received at 27476
full text
mbox
):
From:
Andy Wingo
To:
ludo
Cc:
Ricardo Wurmus
Subject:
Re: libguile/memoize.c is not thread safe,
so syntax parameter expansion is not thread-safe
Date:
Wed, 09 May 2018 10:41:32 +0200
Hi,
On Mon 30 Apr 2018 23:39, ludo
> So the problem, AIUI, is that psyntax evaluates syntax parameters using
> ‘primitive-eval’ (via ‘eval-local-transformer’), but memoization in
> (ice-9 eval) is not thread-safe, hence the random crashes.
Sorry I've been a bit AWOL here... if this diagnosis is correct, then
the problem is ultimately the memoization that needs to be thread-safe,
right?
Is the memoization you are referring to the "set!" in the "lazy" form in
ice-9/eval.scm ? Or something else? FWIW I would not think the "set!"
could be the issue, at least on x86, but who knows.
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Wed, 09 May 2018 09:25:01 GMT)
Full text
and
rfc822 format
available.
Message #84
received at 27476
full text
mbox
):
From:
ludo
To:
Andy Wingo
Cc:
Ricardo Wurmus
Subject:
Re: libguile/memoize.c is not thread safe,
so syntax parameter expansion is not thread-safe
Date:
Wed, 09 May 2018 11:23:59 +0200
Hello Andy!
Andy Wingo
> On Mon 30 Apr 2018 23:39, ludo
>> So the problem, AIUI, is that psyntax evaluates syntax parameters using
>> ‘primitive-eval’ (via ‘eval-local-transformer’), but memoization in
>> (ice-9 eval) is not thread-safe, hence the random crashes.
> Sorry I've been a bit AWOL here... if this diagnosis is correct, then
> the problem is ultimately the memoization that needs to be thread-safe,
> right?
It looks like it, yes.
> Is the memoization you are referring to the "set!" in the "lazy" form in
> ice-9/eval.scm ? Or something else? FWIW I would not think the "set!"
> could be the issue, at least on x86, but who knows.
Actually I’m not sure exactly. ‘memoize-expression’ itself is
side-effect-free, right?
Thanks for your feedback,
Ludo’.
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Wed, 09 May 2018 10:19:02 GMT)
Full text
and
rfc822 format
available.
Message #87
received at 27476
full text
mbox
):
From:
Andy Wingo
To:
ludo
Cc:
Ricardo Wurmus
Subject:
Re: libguile/memoize.c is not thread safe,
so syntax parameter expansion is not thread-safe
Date:
Wed, 09 May 2018 12:18:01 +0200
On Wed 09 May 2018 11:23, ludo
>> Is the memoization you are referring to the "set!" in the "lazy" form in
>> ice-9/eval.scm ? Or something else? FWIW I would not think the "set!"
>> could be the issue, at least on x86, but who knows.
> Actually I’m not sure exactly. ‘memoize-expression’ itself is
> side-effect-free, right?
As far as I know (and I had a look this morning), yes. It takes a
Tree-IL input and returns a memoized output. The internal mutation that
exists in the evaluator is just the lazy "compilation" (see the
invocations of the "lazy" form).
Of course the function being evaluated could mutate shared state as
well!
Andy
Merged
27476
27652
31740
Request was from
ludo
to
control
(Thu, 07 Jun 2018 16:20:03 GMT)
Full text
and
rfc822 format
available.
Merged
27476
27652
31740
34112
Request was from
Ludovic Courtès
to
control
(Tue, 22 Jan 2019 21:00:03 GMT)
Full text
and
rfc822 format
available.
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Tue, 29 Jan 2019 10:24:02 GMT)
Full text
and
rfc822 format
available.
Message #94
received at 27476
full text
mbox
):
From:
Ricardo Wurmus
To:
Ludovic Courtès
Cc:
Andy Wingo
Subject:
Re: bug#27476: guix pull fails on powerful server
Date:
Tue, 29 Jan 2019 11:07:15 +0100
Ludovic Courtès
> ludo
>> I can’t reproduce this with current Guile ‘stable-2.2’, following Andy’s
>> weak-table rewrite¹, so this might have been a weak-table bug showing up
>> under memory pressure.
> With Guile 2.2.3 a similar program triggers a crash very quickly:
> --8<---------------cut here---------------start------------->8---
> $ cat ../guile-debugging/syntax-parms.scm
> (use-modules (ice-9 threads)
> (srfi srfi-1)
> (guix monads)
> (guix store)
> (system base compile))
> (compile #f) ;load modules
> (define threads
> (unfold (lambda (x) (> x 100))
> (lambda (x)
> (call-with-new-thread
> (lambda ()
> (while #t
> (compile
> '(mlet %store-monad ((x y))
> (mbegin %store-monad
> (return x)
> (return y)))
> #:env (current-module)
> #:from 'scheme
> #:to 'tree-il)))))
> 1+
> 0))
> (for-each join-thread threads)
[…]
> --8<---------------cut here---------------end--------------->8---
I tried this with guile 2.2.4 on my laptop with 4 CPUs (according to
lscpu) and I did not get a crash even after waiting for several minutes.
--
Ricardo
Merged
27476
27652
31740
34112
34319
Request was from
Ludovic Courtès
to
control
(Wed, 06 Feb 2019 13:21:02 GMT)
Full text
and
rfc822 format
available.
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Wed, 06 Feb 2019 14:49:02 GMT)
Full text
and
rfc822 format
available.
Message #99
received at 27476
full text
mbox
):
From:
Ludovic Courtès
To:
Andy Wingo
Cc:
Ricardo Wurmus
Subject:
Re: bug#27476: libguile/memoize.c is not thread safe,
so syntax parameter expansion is not thread-safe
Date:
Wed, 06 Feb 2019 15:48:51 +0100
Message part 1
(text/plain, inline)]
Hello Andy!
Since guix-core.drv is the best reproducer I have so far for this syntax
parameter crash, I modified (guix self) to print the name of the files
it’s compiling, and here’s the crash I got (on a 24-core machine):
--8<---------------cut here---------------start------------->8---
building /gnu/store/hf324mhj5607hh2izb01dzhwakmn8am8-guix-core.drv...
[ 39/ 78] loading... 100.0% of 39 filesbuilding "guix/config.scm"
[ 39/ 78] compiling... 0.0% of 39 filesbuilding "guix.scm"
[ 39/ 78] compiling... 0.0% of 39 filesbuilding "guix/monad-repl.scm"
[ 39/ 78] compiling... 0.0% of 39 filesbuilding "guix/store.scm"
[ 39/ 78] compiling... 0.0% of 39 filesbuilding "guix/utils.scm"
[ 39/ 78] compiling... 0.0% of 39 filesbuilding "guix/memoization.scm"
[ 39/ 78] compiling... 0.0% of 39 filesbuilding "guix/profiling.scm"
[ 39/ 78] compiling... 0.0% of 39 filesbuilding "guix/build/utils.scm"
[ 39/ 78] compiling... 0.0% of 39 filesbuilding "guix/build/syscalls.scm"
[ 40/ 78] compiling... 2.6% of 39 filesbuilding "guix/deprecation.scm"
[ 41/ 78] compiling... 5.1% of 39 filesbuilding "guix/i18n.scm"
[ 42/ 78] compiling... 7.7% of 39 filesbuilding "guix/serialization.scm"
[ 43/ 78] compiling... 10.3% of 39 filesbuilding "guix/combinators.scm"
[ 44/ 78] compiling... 12.8% of 39 filesbuilding "guix/monads.scm"
[ 45/ 78] compiling... 15.4% of 39 filesbuilding "guix/records.scm"
[ 46/ 78] compiling... 17.9% of 39 filesIn ice-9/psyntax.scm:
2338:44 19 (expand-let _ _ _ ((line . 447) (column . 6) (filename . "./guix/monads.scm")) _ #
1679:45 18 (parse _ _ _ _ _ _ _)
In ice-9/boot-9.scm:
222:17 17 (map1 (((("placeholder" placeholder) ("l-10a3c941d34314a1-4889" lexical . failure-10a3c941d34314a1-488a) ("placeholder" placeholder) ("placeholder" placeholder) ("l-10a3c9?" . #) ?) ?)))
In ice-9/psyntax.scm:
1409:12 16 (_ _ _ #
2338:44 15 (expand-let _ _ _ ((line . 447) (column . 6) (filename . "./guix/monads.scm")) (hygiene guix monads) #
1679:45 14 (parse _ _ _ _ _ _ _)
In ice-9/boot-9.scm:
222:17 13 (map1 (((("l-10a3c941d34314a1-4894" macro . #
In ice-9/psyntax.scm:
2338:44 12 (expand-let _ _ _ ((line . 447) (column . 6) (filename . "./guix/monads.scm")) (hygiene guix monads) #
1679:45 11 (parse _ _ _ _ _ _ _)
In ice-9/boot-9.scm:
222:17 10 (map1 (((("l-10a3c941d34314a1-48b0" macro . #
In ice-9/psyntax.scm:
2338:44 9 (expand-let _ _ _ ((line . 447) (column . 6) (filename . "./guix/monads.scm")) (hygiene guix monads) #
1612:33 8 (parse (((("placeholder" placeholder) ("l-10a3c941d34314a1-48c8" lexical . tail-10a3c941d34314a1-48c9) ("l-10a3c941d34314a1-48b0" macro . #
1348:32 7 (syntax-type (>>= (mproc head result) (lambda (result) (loop tail result))) (("placeholder" placeholder) ("l-10a3c941d34314a1-48c8" lexical . tail-10a3c941d34314a1-48c9) ("l-?" . #) ?) ?)
1559:32 6 (expand-macro #
In ice-9/boot-9.scm:
752:25 5 (dispatch-exception _ _ _)
751:25 4 (dispatch-exception 1 syntax-error (>>= ">>= (bind) used outside of 'with-monad'" ((line . 451) (column . 9) (filename . "./guix/monads.scm")) (>>= (mproc head result) (lambda # ?)) #))
In guix/build/compile.scm:
122:6 3 (_ _ . _)
In ice-9/boot-9.scm:
829:9 2 (catch #t #
In guix/build/compile.scm:
125:21 1 (_)
In unknown file:
0 (make-stack #t)
guix/build/compile.scm:125:21: Syntax error:
./guix/monads.scm:452:9: >>=: >>= (bind) used outside of 'with-monad' in form (>>= (mproc head result) (lambda (result) (loop tail result)))
builder for `/gnu/store/hf324mhj5607hh2izb01dzhwakmn8am8-guix-core.drv' failed with exit code 1
--8<---------------cut here---------------end--------------->8---
Here (guix monads) was already loaded before, but it’s only when
compiling (guix monads), so after it had been loaded, that we get the
error. The syntax parameter in question is defined in (guix monads)
itself.
I drew the conclusion that our syntax parameter is redefined when we
compile or when we load (guix monads), so there’s a chance that we get
to see the wrong value when we expand (guix monads) (I’m not entirely
sure about the exact sequence of events.)
So I came up with ‘define-syntax-parameter-once’, which is like
‘define-once’ but for syntax parameters (note that we can’t use
‘define-once’ in ‘define-syntax-parameter-once’ because it expands to a
reference to NAME, which doesn’t work for a macro):
Message part 2
(text/x-patch, inline)]
diff --git a/guix/monads.scm b/guix/monads.scm
index 6ae616aca9..1bbf79c8ba 100644
--- a/guix/monads.scm
+++ b/guix/monads.scm
@@ -274,12 +274,20 @@ more optimizations."
(_
#'generic-name))))))))))
-(define-syntax-parameter >>=
+(define-syntax-rule (define-syntax-parameter-once name proc)
+ (eval-when (load eval expand compile)
+ (define name
+ (if (module-locally-bound? (current-module) 'name)
+ (module-ref (current-module) 'name)
+ (make-syntax-transformer 'name 'syntax-parameter
+ (list proc))))))
+(define-syntax-parameter-once >>=
;; The name 'bind' is already taken, so we choose this (obscure) symbol.
(lambda (s)
(syntax-violation '>>= ">>= (bind) used outside of 'with-monad'" s)))
-(define-syntax-parameter return
+(define-syntax-parameter-once return
(lambda (s)
(syntax-violation 'return "return used outside of 'with-monad'" s)))
Message part 3
(text/plain, inline)]
I’ve done a number of rebuilds of guix-core.drv on that 24-core machine
and AFAICS that fixes the issue!
We’ll also have to use it in (guix gexp), which I’m pretty sure will fix
>.
I’ll push this workaround if there are no objections.
On the Guile side, we could maybe arrange to always have ‘define-once’
semantics for those bindings introduced at expansion time, as shown
below (untested):
Message part 4
(text/x-patch, inline)]
--- a/module/ice-9/psyntax.scm
+++ b/module/ice-9/psyntax.scm
@@ -296,9 +296,10 @@
(define put-global-definition-hook
(lambda (symbol type val)
- (module-define! (current-module)
- symbol
- (make-syntax-transformer symbol type val))))
+ (unless (module-locally-bound? (current-module) symbol)
+ (module-define! (current-module)
+ symbol
+ (make-syntax-transformer symbol type val)))))
(define get-global-definition-hook
(lambda (symbol module)
Message part 5
(text/plain, inline)]
WDYT, Andy?
The discussion we had at FOSDEM turned out to be very helpful, thanks
a lot!
Ludo’.
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Wed, 06 Feb 2019 16:15:01 GMT)
Full text
and
rfc822 format
available.
Message #102
received at 27476
full text
mbox
):
From:
Andy Wingo
To:
Ludovic Courtès
Cc:
Ricardo Wurmus
Subject:
Re: bug#27476: libguile/memoize.c is not thread safe,
so syntax parameter expansion is not thread-safe
Date:
Wed, 06 Feb 2019 17:14:37 +0100
Hi!
On Wed 06 Feb 2019 15:48, Ludovic Courtès
> I drew the conclusion that our syntax parameter is redefined when we
> compile or when we load (guix monads), so there’s a chance that we get
> to see the wrong value when we expand (guix monads) (I’m not entirely
> sure about the exact sequence of events.)
You are a wizard!!!!
To be clear, here's the series of events. Firstly, know that defining a
syntax parameter is like:
(define name
(make-syntax-transformer 'name 'syntax-parameter (list f)))
So at the top level you end up with an association between a name and a
"syntax transformer" object (see macros.[ch]). The syntax transformer
object itself consists of its name (for debugging), its syntax type, and
its syntax binding.
For syntax parameters, the binding is a list containing a single
element, the syntax transformer. This list is later used as a key into
a compile-time environment, as it's a unique object associated with the
syntax parameter.
When (syntax-parameterize ((name f*)) exp) is seen, psyntax will look up
`name' in the current expansion-time environment. It asserts that the
name is bound to a syntax transformer and that the syntax transformer is
indeed a syntax parameter, and extracts the associated binding `b'.
Keep in bind that `b' is the single-element list containing the
"default" syntax transformer `f'.
syntax-parameterize then does something weird: it adds an association
between the binding value `b' and `f*' to the expand-time environment.
It does this because the `b' is just a fresh object, so it's a unique
key that's usable for associations. (The way this works is my fault
FWIW.) To be clear, it doesn't add a new definition of `name'; it
instead establishes a new lexical binding for the unique object `b'.
Then when a use of `name' is seen within `exp', Guile finds that `name'
is a syntax parameter, extracts the binding from the syntax transformer
object, then does a second lookup of that binding. If it finds
something bound, it uses that, otherwise it uses the default binding.
I think you see the race here. For an initial state of
(define P (stx-param (list F)))
we have:
thread A thread B
time
resolve P |
extract B |
associate B and F* |
| define P (stx-param (list F**))
resolve P |
extract B (!) |
resolve B (!) |
see F** instead of F* (!) |
> So I came up with ‘define-syntax-parameter-once’, which is like
> ‘define-once’ but for syntax parameters (note that we can’t use
> ‘define-once’ in ‘define-syntax-parameter-once’ because it expands to a
> reference to NAME, which doesn’t work for a macro):
Your fix is good! But, it prevents redefinition of syntax parameters.
I would like to work on a solution that instead of using this
double-lookup, simply adds an association between P and F* in the
environment, instead of doing the double-lookup thing. Probably that
will be 3.0-only.
For 2.2, we can probably update the compiler to trampoline through some
kind of "redefine-syntax" or something that will do (set-car! B F**)
instead of (define P (stx-param B*)). I.e. redefinition keeps the
unique key there.
Andy
Merged
27476
27652
28144
31294
31367
31740
32385
34112
34319
Request was from
Ludovic Courtès
to
control
(Wed, 06 Feb 2019 20:58:03 GMT)
Full text
and
rfc822 format
available.
Information forwarded
to
bug-guix
bug#27476
; Package
guix
(Wed, 06 Feb 2019 22:10:01 GMT)
Full text
and
rfc822 format
available.
Message #107
received at 27476
full text
mbox
):
From:
Ludovic Courtès
To:
Andy Wingo
Cc:
Ricardo Wurmus
Subject:
Re: bug#27476: libguile/memoize.c is not thread safe,
so syntax parameter expansion is not thread-safe
Date:
Wed, 06 Feb 2019 23:09:23 +0100
Hi!
Andy Wingo
> To be clear, here's the series of events. Firstly, know that defining a
> syntax parameter is like:
Thanks for the clear explanation!
>> So I came up with ‘define-syntax-parameter-once’, which is like
>> ‘define-once’ but for syntax parameters (note that we can’t use
>> ‘define-once’ in ‘define-syntax-parameter-once’ because it expands to a
>> reference to NAME, which doesn’t work for a macro):
> Your fix is good! But, it prevents redefinition of syntax parameters.
Yes. It’s acceptable in this case, so I’ve pushed it as a workaround as
commit 8245bb74fc7bdcdc2f9d458057cefc9cd982e489 in Guix.
> I would like to work on a solution that instead of using this
> double-lookup, simply adds an association between P and F* in the
> environment, instead of doing the double-lookup thing. Probably that
> will be 3.0-only.
> For 2.2, we can probably update the compiler to trampoline through some
> kind of "redefine-syntax" or something that will do (set-car! B F**)
> instead of (define P (stx-param B*)). I.e. redefinition keeps the
> unique key there.
Sounds good. Are you taking a look at this?
Perhaps that’d be a good excuse to release 2.2.5.
Thank you!
Ludo’.
bug reassigned from package '
guix
' to '
guile
'.
Request was from
Ludovic Courtès
to
control
(Sat, 09 Feb 2019 22:12:01 GMT)
Full text
and
rfc822 format
available.
Reply sent
to
Ludovic Courtès
You have taken responsibility.
(Thu, 17 Dec 2020 15:13:01 GMT)
Full text
and
rfc822 format
available.
Notification sent
to
Leo Famulari
bug acknowledged by developer.
(Thu, 17 Dec 2020 15:13:01 GMT)
Full text
and
rfc822 format
available.
Message #114
received at 27476-done
full text
mbox
):
From:
Ludovic Courtès
To:
Andy Wingo
Cc:
27476-done
Subject:
Re: bug#27476: libguile/memoize.c is not thread safe, so syntax
parameter expansion is not thread-safe
Date:
Thu, 17 Dec 2020 16:12:17 +0100
Hi!
For the record, this was fixed in
61a8c9300daeb730fe5094f889bf13241942be84, which made it into 2.9/3.0,
and 2dccec9f553776656d9378e2315ad32d2e55286b, which made it into 2.2.5.
Closing! :-)
Ludo’.
Reply sent
to
Ludovic Courtès
You have taken responsibility.
(Thu, 17 Dec 2020 15:13:02 GMT)
Full text
and
rfc822 format
available.
Notification sent
to
Joshua Sierles
bug acknowledged by developer.
(Thu, 17 Dec 2020 15:13:02 GMT)
Full text
and
rfc822 format
available.
Reply sent
to
Ludovic Courtès
You have taken responsibility.
(Thu, 17 Dec 2020 15:13:02 GMT)
Full text
and
rfc822 format
available.
Notification sent
to
Christopher Baines
bug acknowledged by developer.
(Thu, 17 Dec 2020 15:13:02 GMT)
Full text
and
rfc822 format
available.
Reply sent
to
Ludovic Courtès
You have taken responsibility.
(Thu, 17 Dec 2020 15:13:02 GMT)
Full text
and
rfc822 format
available.
Notification sent
to
Fis Trivial
bug acknowledged by developer.
(Thu, 17 Dec 2020 15:13:02 GMT)
Full text
and
rfc822 format
available.
Reply sent
to
Ludovic Courtès
You have taken responsibility.
(Thu, 17 Dec 2020 15:13:02 GMT)
Full text
and
rfc822 format
available.
Notification sent
to
George myglc2 Clemmer
bug acknowledged by developer.
(Thu, 17 Dec 2020 15:13:03 GMT)
Full text
and
rfc822 format
available.
Reply sent
to
Ludovic Courtès
You have taken responsibility.
(Thu, 17 Dec 2020 15:13:03 GMT)
Full text
and
rfc822 format
available.
Notification sent
to
Fis Trivial
bug acknowledged by developer.
(Thu, 17 Dec 2020 15:13:03 GMT)
Full text
and
rfc822 format
available.
Reply sent
to
Ludovic Courtès
You have taken responsibility.
(Thu, 17 Dec 2020 15:13:03 GMT)
Full text
and
rfc822 format
available.
Notification sent
to
Chris Marusich
bug acknowledged by developer.
(Thu, 17 Dec 2020 15:13:03 GMT)
Full text
and
rfc822 format
available.
Reply sent
to
Ludovic Courtès
You have taken responsibility.
(Thu, 17 Dec 2020 15:13:03 GMT)
Full text
and
rfc822 format
available.
Notification sent
to
Mark H Weaver
bug acknowledged by developer.
(Thu, 17 Dec 2020 15:13:03 GMT)
Full text
and
rfc822 format
available.
Reply sent
to
Ludovic Courtès
You have taken responsibility.
(Thu, 17 Dec 2020 15:13:04 GMT)
Full text
and
rfc822 format
available.
Notification sent
to
Mark H Weaver
bug acknowledged by developer.
(Thu, 17 Dec 2020 15:13:04 GMT)
Full text
and
rfc822 format
available.
bug archived.
Request was from
Debbugs Internal Request
to
internal_control
(Fri, 15 Jan 2021 12:24:04 GMT)
Full text
and
rfc822 format
available.
This bug report was last modified 5 years and 100 days ago.
Previous
Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.