Project:Toolchain/time64 migration - Gentoo wiki

Project:Toolchain/time64 migration - Gentoo wiki
Jump to:
content
Project:Toolchain/time64 migration
From Gentoo Wiki
Project:Toolchain
Jump to:navigation
Jump to:search
Traditional
time_t
is 32-bit and is subject to the
Y2038 problem
. glibc 2.34 onwards supports
opting-in
to a 64-bit
time_t
which allows handling time beyond Y2038.
This page discusses how to handle the migration for (32-bit) systems affected by this, including non-glibc environments. 64-bit systems are not affected.
Mixing 32-bit
time_t
and 64-bit
time_t
is a recipe for disaster where
time_t
is exposed in a public API and the consumer/provider are mismatched e.g.
net-misc/wget
with
net-libs/gnutls
in
bug #828001
. This is actually
a problem with Large File Support (LFS) too but it seems to happen far less frequently there, or it was minimised by LARGEFILE_SOURCE allowing dual ABI in packages.
For 32-bit arches, we need to coordinate flipping glibc systems over to use 64-bit
time_t
using
CPPFLAGS="-D_TIME_BITS=64"
(note that not all packages may support/respect
CPPFLAGS
). Again, 64-bit arches are already fine.
Contents
Problem
Notes
Plans
3.1
TODO
musl
LFS
Possible solutions
6.1
Rebuild after profile change
6.2
Revbump ebuilds
6.3
time64 USE flag
6.4
Summary
See also
External resources
References
Problem
Sneaky bits:
Libraries using time_t
directly or indirectly
needs to be switched at the same time. Applications using any such libraries must be switched at the same time too.
Software might be using
time_t
(and e.g.
off_t
, needing Large File Support (LFS)) without it being obvious. For example, it might be handling a
stat
struct which contains both
off_t
and
st_atim
Software needs to provide a way to stick with 32-bit
time_t
for now to avoid making the problem worse, but allow optionally using 64-bit
time_t
We likely have to complete
Modern C porting
first to remove any instances of
-Wimplicit-function-declaration
otherwise the redirects in glibc for e.g. time->time64
won't actually work
. We
might
be able to avoid this with some hacks like removing the 32-bit symbols entirely from glibc so we get linker errors.
The core issues are:
Auditing packages piecemeal is an unfeasibly large amount of work
glibc has no mechanism to hard-switch yet (and discussion has stalled
[1]
Acting alone in a hard-switch (or at all, really) risks incompatibility with other distributions, as all of this obviously affects ABI
time_t
or
time_t
-dependent types in a library's API is not always straightforward to see
Notes
Misc other notes:
sam summarised the situation on
libc-alpha in November
in "On time64 and Large File Support" which mentions the problems we face
dilfridge
rebooted
the discussion in January in "The time64 and Large File Support mess"
Support is new in glibc 2.34, stabled in April 2022 in
bug #833191
gnulib automatically enables time64 support if the system supports it, breaking ABI
edit
: gnulib
reverted
this). We set
gl_cv_type_time_t_bits_macro=no
(and may need to set
ac_cv_type_time_t_y2038=no
) for now (for software released with the intermediate "bad" versions of gnulib).
See older discussion on libc-alpha
[2]
Packages in Gentoo should use
tc-has-64bit-time_t
if they need to detect the current size of time_t
Packages should generally feel free to enable LFS and 64-bit time_t if upstream encourage it or default to it
as long as no relevant types appear on an ABI boundary
(if you're not sure, or don't know: don't do it)
Libraries and those with such types on an ABI boundary must disable it instead until the global migration is done
Either way, it shouldn't be done based on a USE flag
Our friends in opensuse are on it too (
Gentoo's tracker bug:
bug #876883
mgorny's blog posts:
The perils of transition to 64-bit time_t
Testing the safe time64 transition path
Plans
TODO
Push for an autoconf release which both fixes some
Modern C porting
issues and contains the needed macros for Y2038 support. The needed fixes are already in git.
Agree on a way with the upstream glibc + distro community on how to hard-switch glibc.
Coordinate hard-switching once (plenty of!) testing has been done
Achieve consensus on new tuple names for time64'd ABIs
Gentoo-specific TODO items, but they may affect other distros too:
Implement LFS and time64 QA check for Portage
bug #549092
Rebased version of LFS check at
Draft version of time64 check at
Needs
Backport autoconf
commit
from master to make
AC_SYS_LARGEFILE
imply 64-bit time_t
(this got yanked upstream so no need to just accept it now)
How do we safely recommend rebuilding?
Start with (some subset of?) @system then @world?
How do we identify mismatched systems (possibly musl)? Will they just need to follow the glibc procedure we decide on (sans flags)?
We need a way to supply binary packages with time32 and time64 configurations (USE flag?).
musl
musl upstream already migrated to pure 64-bit
time_t
in 1.2.0
Upstream have a detailed set of notes
[3]
on the problem
Adélie Linux's time64 notes
Gentoo specific notes:
Need to check how/if at all we handled it in Gentoo
Check whether there's more work to be done to ensure user systems are in a consistent state
1.2.0 was
added
on 2020-02-25 and 1.2.1 (first of 1.2.x) was
stabled
on 2020-08-20.
Possible user systems are in an inconsistent state given we've not rebuilt / news item'd for this?
LFS
We may want to compare with how Large File Support (LFS) /
FILE_OFFSET_BITS
has been handled.
Much
more mature support overall in the ecosystem, although it wasn't generally treated like the ABI break it was.
Misc notes:
For glibc, time64 needs LFS (example of why: see e.g. stat struct).
Meson already adds this as required
autoconf has
AC_SYS_LARGEFILE
which things
should
be using (but may not be!)
Gentoo specific notes:
bug #471102
tracks progress in Gentoo overall.
We have
append-lfs-flags
(and
filter-lfs-flags
) in flag-o-matic.eclass
Possible solutions
Rebuild after profile change
Note
Not everyone is using autoconf or even passing flags correctly. A safer, though probably more controversial way would be to force changed defaults in gcc and glibc.
We need to agree on names for these new ABIs, ideally using new tuple names, and do so with other distributions
In current profiles, set
gl_cv_type_time_t_bits_macro=no
to prevent gnulib autoconf macro from automatically adding
-D_TIME_BITS=64
done
, note that gnulib reverted the change (see above))
Ditto for
ac_cv_type_time_t_y2038
In new profile, add
CPPFLAGS="-D_FILE_OFFSET_BITS=64 -D_TIME_BITS=64"
do CFLAGS instead given insufficient packages respect CPPFLAGS
).
Advise users to adjust custom
CPPFLAGS
if necessary, and rebuild @world after switching profiles.
Some more thoughts / suggestions:
Use as ABI suffix t64, e.g., i686-linux-gnut64
This will require fixing some packages (notably, ncurses and clang)
clang patch
; but we need to figure out backports (Triple changes affect ABI)
Both CHOST and CHOST_x86 need to be patched (else e.g. clang uses the wrong tool prefix)
We already have autoconf magic in current profiles, that needs to be overridden then (where?)
Unfortunately, even default CFLAGS may not be good enough.
Add a force-time64 useflag to gcc,
example patch switching the defaults
force the defines into CPPFLAGS in the compiler if the useflag is set?!
A safer transition route is possible by copying existing libraries to
libt32
and adding RUNPATHs to all existing executables
implemented in
time64-prep
tool
Revbump ebuilds
Warning
This option is considered far too much work and doesn't achieve ABI compatibility with other distros, so is unlikely to be pursued.
In packages where
time_t
is utilized, revbump and add
append-cppflags -D_FILE_OFFSET_BITS=64 -D_TIME_BITS=64
This doesn't ensure that libraries and their consumers get rebuilt together, but it should work out in the end.
Presumably, we'd have a large
package.mask
for this.
time64 USE flag
Warning
Not likely to be pursued for the same reasons as "Revbump ebuilds".
Add
USE=time64
masked everywhere
May need lots of
eautoreconf
(after patching autoconf)
Lots of --enable-year2038 added to econfs
Unmask on to-be-created
features/time64
profile
Allows expressing dependencies on time64ness to help rebuild order
Allows adding machinery to ebuilds at our own pace, then create new profiles when ready
Doesn't cause churn to 64-bit arches
Summary
Solution
Consistency during migration
Minimises disruption to bystander/unaffected users (64-bit arches)
Amount of manual labor needed
Avoids flag day
External or historical binary compatibility
Rebuild after profile change (just changes cache variables)
Provides no guarantees of correct rebuild order and may require several rebuilds to ensure consistent ABI.
The change is invisible to these users as they already have 64-bit support and won't be changing profiles.
Not much at all wrt packaging minutiae (of course, there's still bug fixing.)
No
No
New profiles, new CHOST
Provides no guarantees of correct rebuild order and may require several rebuilds to ensure consistent ABI.
The change is invisible to these users as they already have 64-bit support and won't be changing profiles.
Not much at all wrt packaging minutiae (of course, there's still bug fixing). Need to make sure the new CHOST doesnt break anything.
Users can migrate at leisure
CHOST change cleanly separates ABI
Revbump ebuilds to add time64 CFLAGS
Provides no guarantees of correct rebuild order and may require several rebuilds to ensure consistent ABI.
All users, even those unaffected, have to rebuild.
Lots of ebuild churn.
We can make the changes gradually, by traversing leaf packages, and building up to libraries.
No
time64 USE flag
If dependencies are expressed correctly, we can ensure libraries gain time64 support before consumers, and we can use use= deps to enforce correctness.
The USE flag will be masked on irrelevant profiles so only
--newuse
-N
) users will be affected.
A lot of work modifying
many
ebuilds, especially if we're going to get the dependencies right, which is the advantage from this option...
We'd need a flag day to eventually remove the time64 flag, but not to add it.
No
See also
Modern C porting
— catalogs the requirements that older
software must now meet in order to correctly build with modern compilers, and includes explanations and tips on how to
port older codebases to modern C
External resources
Preparing for Y2038 (Already?!)
glibc is still not Y2038 compliant by default
Debian discusses how to handle 2038
(from 2020, outdated wrt glibc)
Debian wiki: 64bit-time
References
Sam James.
On time64 and Large File Support
libc-alpha mailing list
, November 11th, 2022. Retrieved on January 19th, 2023.
Kaz Kylheku.
Best practices in regard to -D_TIME_BITS=64
libc-alpha mailing list
, January 5th, 2022. Retrieved on January 19th, 2023.
Rich Felker.
musl time64 Release Notes
musl libc project
, February 20th, 2020. Retrieved on January 19th, 2023.
Retrieved from "
Category
Porting notes