1. 22 Jun, 2022 40 commits
    • Jason A. Donenfeld's avatar
      random: move randomize_page() into mm where it belongs · 1aeedbe0
      Jason A. Donenfeld authored
      commit 5ad7dd88
      
       upstream.
      
      randomize_page is an mm function. It is documented like one. It contains
      the history of one. It has the naming convention of one. It looks
      just like another very similar function in mm, randomize_stack_top().
      And it has always been maintained and updated by mm people. There is no
      need for it to be in random.c. In the "which shape does not look like
      the other ones" test, pointing to randomize_page() is correct.
      
      So move randomize_page() into mm/util.c, right next to the similar
      randomize_stack_top() function.
      
      This commit contains no actual code changes.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      1aeedbe0
    • Jason A. Donenfeld's avatar
      random: move initialization functions out of hot pages · ceaf1fee
      Jason A. Donenfeld authored
      commit 560181c2
      
       upstream.
      
      Much of random.c is devoted to initializing the rng and accounting for
      when a sufficient amount of entropy has been added. In a perfect world,
      this would all happen during init, and so we could mark these functions
      as __init. But in reality, this isn't the case: sometimes the rng only
      finishes initializing some seconds after system init is finished.
      
      For this reason, at the moment, a whole host of functions that are only
      used relatively close to system init and then never again are intermixed
      with functions that are used in hot code all the time. This creates more
      cache misses than necessary.
      
      In order to pack the hot code closer together, this commit moves the
      initialization functions that can't be marked as __init into
      .text.unlikely by way of the __cold attribute.
      
      Of particular note is moving credit_init_bits() into a macro wrapper
      that inlines the crng_ready() static branch check. This avoids a
      function call to a nop+ret, and most notably prevents extra entropy
      arithmetic from being computed in mix_interrupt_randomness().
      Reviewed-by: default avatarDominik Brodowski <linux@dominikbrodowski.net>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      ceaf1fee
    • Jason A. Donenfeld's avatar
      random: make consistent use of buf and len · d3bf98d6
      Jason A. Donenfeld authored
      commit a1940263
      
       upstream.
      
      The current code was a mix of "nbytes", "count", "size", "buffer", "in",
      and so forth. Instead, let's clean this up by naming input parameters
      "buf" (or "ubuf") and "len", so that you always understand that you're
      reading this variety of function argument.
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      d3bf98d6
    • Jason A. Donenfeld's avatar
      random: use proper return types on get_random_{int,long}_wait() · 70fce7f1
      Jason A. Donenfeld authored
      commit 7c3a8a1d
      
       upstream.
      
      Before these were returning signed values, but the API is intended to be
      used with unsigned values.
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      70fce7f1
    • Jason A. Donenfeld's avatar
      random: remove extern from functions in header · d05948dc
      Jason A. Donenfeld authored
      commit 7782cfec
      
       upstream.
      
      Accoriding to the kernel style guide, having `extern` on functions in
      headers is old school and deprecated, and doesn't add anything. So remove
      them from random.h, and tidy up the file a little bit too.
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      d05948dc
    • Jason A. Donenfeld's avatar
      random: use static branch for crng_ready() · d8b42964
      Jason A. Donenfeld authored
      commit f5bda35f
      
       upstream.
      
      Since crng_ready() is only false briefly during initialization and then
      forever after becomes true, we don't need to evaluate it after, making
      it a prime candidate for a static branch.
      
      One complication, however, is that it changes state in a particular call
      to credit_init_bits(), which might be made from atomic context, which
      means we must kick off a workqueue to change the static key. Further
      complicating things, credit_init_bits() may be called sufficiently early
      on in system initialization such that system_wq is NULL.
      
      Fortunately, there exists the nice function execute_in_process_context(),
      which will immediately execute the function if !in_interrupt(), and
      otherwise defer it to a workqueue. During early init, before workqueues
      are available, in_interrupt() is always false, because interrupts
      haven't even been enabled yet, which means the function in that case
      executes immediately. Later on, after workqueues are available,
      in_interrupt() might be true, but in that case, the work is queued in
      system_wq and all goes well.
      
      Cc: Theodore Ts'o <tytso@mit.edu>
      Cc: Sultan Alsawaf <sultan@kerneltoast.com>
      Reviewed-by: default avatarDominik Brodowski <linux@dominikbrodowski.net>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      d8b42964
    • Jason A. Donenfeld's avatar
      random: credit architectural init the exact amount · 087a14b9
      Jason A. Donenfeld authored
      commit 12e45a2a
      
       upstream.
      
      RDRAND and RDSEED can fail sometimes, which is fine. We currently
      initialize the RNG with 512 bits of RDRAND/RDSEED. We only need 256 bits
      of those to succeed in order to initialize the RNG. Instead of the
      current "all or nothing" approach, actually credit these contributions
      the amount that is actually contributed.
      Reviewed-by: default avatarDominik Brodowski <linux@dominikbrodowski.net>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      087a14b9
    • Jason A. Donenfeld's avatar
      random: handle latent entropy and command line from random_init() · ac48f7be
      Jason A. Donenfeld authored
      commit 2f14062b
      
       upstream.
      
      Currently, start_kernel() adds latent entropy and the command line to
      the entropy bool *after* the RNG has been initialized, deferring when
      it's actually used by things like stack canaries until the next time
      the pool is seeded. This surely is not intended.
      
      Rather than splitting up which entropy gets added where and when between
      start_kernel() and random_init(), just do everything in random_init(),
      which should eliminate these kinds of bugs in the future.
      
      While we're at it, rename the awkwardly titled "rand_initialize()" to
      the more standard "random_init()" nomenclature.
      Reviewed-by: default avatarDominik Brodowski <linux@dominikbrodowski.net>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      ac48f7be
    • Jason A. Donenfeld's avatar
      random: use proper jiffies comparison macro · 736a2264
      Jason A. Donenfeld authored
      commit 8a5b8a4a
      
       upstream.
      
      This expands to exactly the same code that it replaces, but makes things
      consistent by using the same macro for jiffy comparisons throughout.
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      736a2264
    • Jason A. Donenfeld's avatar
      random: remove ratelimiting for in-kernel unseeded randomness · 3266fba2
      Jason A. Donenfeld authored
      commit cc1e127b
      
       upstream.
      
      The CONFIG_WARN_ALL_UNSEEDED_RANDOM debug option controls whether the
      kernel warns about all unseeded randomness or just the first instance.
      There's some complicated rate limiting and comparison to the previous
      caller, such that even with CONFIG_WARN_ALL_UNSEEDED_RANDOM enabled,
      developers still don't see all the messages or even an accurate count of
      how many were missed. This is the result of basically parallel
      mechanisms aimed at accomplishing more or less the same thing, added at
      different points in random.c history, which sort of compete with the
      first-instance-only limiting we have now.
      
      It turns out, however, that nobody cares about the first unseeded
      randomness instance of in-kernel users. The same first user has been
      there for ages now, and nobody is doing anything about it. It isn't even
      clear that anybody _can_ do anything about it. Most places that can do
      something about it have switched over to using get_random_bytes_wait()
      or wait_for_random_bytes(), which is the right thing to do, but there is
      still much code that needs randomness sometimes during init, and as a
      geeneral rule, if you're not using one of the _wait functions or the
      readiness notifier callback, you're bound to be doing it wrong just
      based on that fact alone.
      
      So warning about this same first user that can't easily change is simply
      not an effective mechanism for anything at all. Users can't do anything
      about it, as the Kconfig text points out -- the problem isn't in
      userspace code -- and kernel developers don't or more often can't react
      to it.
      
      Instead, show the warning for all instances when CONFIG_WARN_ALL_UNSEEDED_RANDOM
      is set, so that developers can debug things need be, or if it isn't set,
      don't show a warning at all.
      
      At the same time, CONFIG_WARN_ALL_UNSEEDED_RANDOM now implies setting
      random.ratelimit_disable=1 on by default, since if you care about one
      you probably care about the other too. And we can clean up usage around
      the related urandom_warning ratelimiter as well (whose behavior isn't
      changing), so that it properly counts missed messages after the 10
      message threshold is reached.
      
      Cc: Theodore Ts'o <tytso@mit.edu>
      Cc: Dominik Brodowski <linux@dominikbrodowski.net>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      3266fba2
    • Jason A. Donenfeld's avatar
      random: move initialization out of reseeding hot path · c5373bd6
      Jason A. Donenfeld authored
      commit 68c9c8b1
      
       upstream.
      
      Initialization happens once -- by way of credit_init_bits() -- and then
      it never happens again. Therefore, it doesn't need to be in
      crng_reseed(), which is a hot path that is called multiple times. It
      also doesn't make sense to have there, as initialization activity is
      better associated with initialization routines.
      
      After the prior commit, crng_reseed() now won't be called by multiple
      concurrent callers, which means that we can safely move the
      "finialize_init" logic into crng_init_bits() unconditionally.
      Reviewed-by: default avatarDominik Brodowski <linux@dominikbrodowski.net>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      c5373bd6
    • Jason A. Donenfeld's avatar
      random: avoid initializing twice in credit race · 0747ad15
      Jason A. Donenfeld authored
      commit fed7ef06
      
       upstream.
      
      Since all changes of crng_init now go through credit_init_bits(), we can
      fix a long standing race in which two concurrent callers of
      credit_init_bits() have the new bit count >= some threshold, but are
      doing so with crng_init as a lower threshold, checked outside of a lock,
      resulting in crng_reseed() or similar being called twice.
      
      In order to fix this, we can use the original cmpxchg value of the bit
      count, and only change crng_init when the bit count transitions from
      below a threshold to meeting the threshold.
      Reviewed-by: default avatarDominik Brodowski <linux@dominikbrodowski.net>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      0747ad15
    • Jason A. Donenfeld's avatar
      random: use symbolic constants for crng_init states · 0baeec0e
      Jason A. Donenfeld authored
      commit e3d2c5e7
      
       upstream.
      
      crng_init represents a state machine, with three states, and various
      rules for transitions. For the longest time, we've been managing these
      with "0", "1", and "2", and expecting people to figure it out. To make
      the code more obvious, replace these with proper enum values
      representing the transition, and then redocument what each of these
      states mean.
      Reviewed-by: default avatarDominik Brodowski <linux@dominikbrodowski.net>
      Cc: Joe Perches <joe@perches.com>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      0baeec0e
    • Jason A. Donenfeld's avatar
      siphash: use one source of truth for siphash permutations · 55d64df3
      Jason A. Donenfeld authored
      commit e73aaae2
      
       upstream.
      
      The SipHash family of permutations is currently used in three places:
      
      - siphash.c itself, used in the ordinary way it was intended.
      - random32.c, in a construction from an anonymous contributor.
      - random.c, as part of its fast_mix function.
      
      Each one of these places reinvents the wheel with the same C code, same
      rotation constants, and same symmetry-breaking constants.
      
      This commit tidies things up a bit by placing macros for the
      permutations and constants into siphash.h, where each of the three .c
      users can access them. It also leaves a note dissuading more users of
      them from emerging.
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      55d64df3
    • Jason A. Donenfeld's avatar
      random: help compiler out with fast_mix() by using simpler arguments · e4e8a9f8
      Jason A. Donenfeld authored
      commit 791332b3
      
       upstream.
      
      Now that fast_mix() has more than one caller, gcc no longer inlines it.
      That's fine. But it also doesn't handle the compound literal argument we
      pass it very efficiently, nor does it handle the loop as well as it
      could. So just expand the code to spell out this function so that it
      generates the same code as it did before. Performance-wise, this now
      behaves as it did before the last commit. The difference in actual code
      size on x86 is 45 bytes, which is less than a cache line.
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      e4e8a9f8
    • Jason A. Donenfeld's avatar
      random: do not use input pool from hard IRQs · bf3b51eb
      Jason A. Donenfeld authored
      commit e3e33fc2
      
       upstream.
      
      Years ago, a separate fast pool was added for interrupts, so that the
      cost associated with taking the input pool spinlocks and mixing into it
      would be avoided in places where latency is critical. However, one
      oversight was that add_input_randomness() and add_disk_randomness()
      still sometimes are called directly from the interrupt handler, rather
      than being deferred to a thread. This means that some unlucky interrupts
      will be caught doing a blake2s_compress() call and potentially spinning
      on input_pool.lock, which can also be taken by unprivileged users by
      writing into /dev/urandom.
      
      In order to fix this, add_timer_randomness() now checks whether it is
      being called from a hard IRQ and if so, just mixes into the per-cpu IRQ
      fast pool using fast_mix(), which is much faster and can be done
      lock-free. A nice consequence of this, as well, is that it means hard
      IRQ context FPU support is likely no longer useful.
      
      The entropy estimation algorithm used by add_timer_randomness() is also
      somewhat different than the one used for add_interrupt_randomness(). The
      former looks at deltas of deltas of deltas, while the latter just waits
      for 64 interrupts for one bit or for one second since the last bit. In
      order to bridge these, and since add_interrupt_randomness() runs after
      an add_timer_randomness() that's called from hard IRQ, we add to the
      fast pool credit the related amount, and then subtract one to account
      for add_interrupt_randomness()'s contribution.
      
      A downside of this, however, is that the num argument is potentially
      attacker controlled, which puts a bit more pressure on the fast_mix()
      sponge to do more than it's really intended to do. As a mitigating
      factor, the first 96 bits of input aren't attacker controlled (a cycle
      counter followed by zeros), which means it's essentially two rounds of
      siphash rather than one, which is somewhat better. It's also not that
      much different from add_interrupt_randomness()'s use of the irq stack
      instruction pointer register.
      
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Filipe Manana <fdmanana@suse.com>
      Cc: Peter Zijlstra <peterz@infradead.org>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: Theodore Ts'o <tytso@mit.edu>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      bf3b51eb
    • Jason A. Donenfeld's avatar
      random: order timer entropy functions below interrupt functions · 6d4203a2
      Jason A. Donenfeld authored
      commit a4b5c26b
      
       upstream.
      
      There are no code changes here; this is just a reordering of functions,
      so that in subsequent commits, the timer entropy functions can call into
      the interrupt ones.
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      6d4203a2
    • Jason A. Donenfeld's avatar
      random: do not pretend to handle premature next security model · 58da574f
      Jason A. Donenfeld authored
      commit e85c0fc1 upstream.
      
      Per the thread linked below, "premature next" is not considered to be a
      realistic threat model, and leads to more serious security problems.
      
      "Premature next" is the scenario in which:
      
      - Attacker compromises the current state of a fully initialized RNG via
        some kind of infoleak.
      - New bits of entropy are added directly to the key used to generate the
        /dev/urandom stream, without any buffering or pooling.
      - Attacker then, somehow having read access to /dev/urandom, samples RNG
        output and brute forces the individual new bits that were added.
      - Result: the RNG never "recovers" from the initial compromise, a
        so-called violation of what academics term "post-compromise security".
      
      The usual solutions to this involve some form of delaying when entropy
      gets mixed into the crng. With Fortuna, this involves multiple input
      buckets. With what the Linux RNG was trying to do prior, this involves
      entropy estimation.
      
      However, by delaying when entropy gets mixed in, it also means that RNG
      compromises are extremely dangerous during the window of time before
      the RNG has gathered enough entropy, during which time nonces may become
      predictable (or repeated), ephemeral keys may not be secret, and so
      forth. Moreover, it's unclear how realistic "premature next" is from an
      attack perspective, if these attacks even make sense in practice.
      
      Put together -- and discussed in more detail in the thread below --
      these constitute grounds for just doing away with the current code that
      pretends to handle premature next. I say "pretends" because it wasn't
      doing an especially great job at it either; should we change our mind
      about this direction, we would probably implement Fortuna to "fix" the
      "problem", in which case, removing the pretend solution still makes
      sense.
      
      This also reduces the crng reseed period from 5 minutes down to 1
      minute. The rationale from the thread might lead us toward reducing that
      even further in the future (or even eliminating it), but that remains a
      topic of a future commit.
      
      At a high level, this patch changes semantics from:
      
          Before: Seed for the first time after 256 "bits" of estimated
          entropy have been accumulated since the system booted. Thereafter,
          reseed once every five minutes, but only if 256 new "bits" have been
          accumulated since the last reseeding.
      
          After: Seed for the first time after 256 "bits" of estimated entropy
          have been accumulated since the system booted. Thereafter, reseed
          once every minute.
      
      Most of this patch is renaming and removing: POOL_MIN_BITS becomes
      POOL_INIT_BITS, credit_entropy_bits() becomes credit_init_bits(),
      crng_reseed() loses its "force" parameter since it's now always true,
      the drain_entropy() function no longer has any use so it's removed,
      entropy estimation is skipped if we've already init'd, the various
      notifiers for "low on entropy" are now only active prior to init, and
      finally, some documentation comments are cleaned up here and there.
      
      Link: https://lore.kernel.org/lkml/YmlMGx6+uigkGiZ0@zx2c4.com/
      
      
      Cc: Theodore Ts'o <tytso@mit.edu>
      Cc: Nadia Heninger <nadiah@cs.ucsd.edu>
      Cc: Tom Ristenpart <ristenpart@cornell.edu>
      Reviewed-by: default avatarEric Biggers <ebiggers@google.com>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      58da574f
    • Jason A. Donenfeld's avatar
      random: use first 128 bits of input as fast init · e739d5bd
      Jason A. Donenfeld authored
      commit 5c3b747e
      
       upstream.
      
      Before, the first 64 bytes of input, regardless of how entropic it was,
      would be used to mutate the crng base key directly, and none of those
      bytes would be credited as having entropy. Then 256 bits of credited
      input would be accumulated, and only then would the rng transition from
      the earlier "fast init" phase into being actually initialized.
      
      The thinking was that by mixing and matching fast init and real init, an
      attacker who compromised the fast init state, considered easy to do
      given how little entropy might be in those first 64 bytes, would then be
      able to bruteforce bits from the actual initialization. By keeping these
      separate, bruteforcing became impossible.
      
      However, by not crediting potentially creditable bits from those first 64
      bytes of input, we delay initialization, and actually make the problem
      worse, because it means the user is drawing worse random numbers for a
      longer period of time.
      
      Instead, we can take the first 128 bits as fast init, and allow them to
      be credited, and then hold off on the next 128 bits until they've
      accumulated. This is still a wide enough margin to prevent bruteforcing
      the rng state, while still initializing much faster.
      
      Then, rather than trying to piecemeal inject into the base crng key at
      various points, instead just extract from the pool when we need it, for
      the crng_init==0 phase. Performance may even be better for the various
      inputs here, since there are likely more calls to mix_pool_bytes() then
      there are to get_random_bytes() during this phase of system execution.
      
      Since the preinit injection code is gone, bootloader randomness can then
      do something significantly more straight forward, removing the weird
      system_wq hack in hwgenerator randomness.
      
      Cc: Theodore Ts'o <tytso@mit.edu>
      Cc: Dominik Brodowski <linux@dominikbrodowski.net>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      e739d5bd
    • Jason A. Donenfeld's avatar
      random: do not use batches when !crng_ready() · c44f8b38
      Jason A. Donenfeld authored
      commit cbe89e5a
      
       upstream.
      
      It's too hard to keep the batches synchronized, and pointless anyway,
      since in !crng_ready(), we're updating the base_crng key really often,
      where batching only hurts. So instead, if the crng isn't ready, just
      call into get_random_bytes(). At this stage nothing is performance
      critical anyhow.
      
      Cc: Theodore Ts'o <tytso@mit.edu>
      Reviewed-by: default avatarDominik Brodowski <linux@dominikbrodowski.net>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      c44f8b38
    • Jason A. Donenfeld's avatar
      random: insist on random_get_entropy() existing in order to simplify · e247ea8d
      Jason A. Donenfeld authored
      commit 4b758eda
      
       upstream.
      
      All platforms are now guaranteed to provide some value for
      random_get_entropy(). In case some bug leads to this not being so, we
      print a warning, because that indicates that something is really very
      wrong (and likely other things are impacted too). This should never be
      hit, but it's a good and cheap way of finding out if something ever is
      problematic.
      
      Since we now have viable fallback code for random_get_entropy() on all
      platforms, which is, in the worst case, not worse than jiffies, we can
      count on getting the best possible value out of it. That means there's
      no longer a use for using jiffies as entropy input. It also means we no
      longer have a reason for doing the round-robin register flow in the IRQ
      handler, which was always of fairly dubious value.
      
      Instead we can greatly simplify the IRQ handler inputs and also unify
      the construction between 64-bits and 32-bits. We now collect the cycle
      counter and the return address, since those are the two things that
      matter. Because the return address and the irq number are likely
      related, to the extent we mix in the irq number, we can just xor it into
      the top unchanging bytes of the return address, rather than the bottom
      changing bytes of the cycle counter as before. Then, we can do a fixed 2
      rounds of SipHash/HSipHash. Finally, we use the same construction of
      hashing only half of the [H]SipHash state on 32-bit and 64-bit. We're
      not actually discarding any entropy, since that entropy is carried
      through until the next time. And more importantly, it lets us do the
      same sponge-like construction everywhere.
      
      Cc: Theodore Ts'o <tytso@mit.edu>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      e247ea8d
    • Jason A. Donenfeld's avatar
      xtensa: use fallback for random_get_entropy() instead of zero · 9bfbcb37
      Jason A. Donenfeld authored
      commit e10e2f58
      
       upstream.
      
      In the event that random_get_entropy() can't access a cycle counter or
      similar, falling back to returning 0 is really not the best we can do.
      Instead, at least calling random_get_entropy_fallback() would be
      preferable, because that always needs to return _something_, even
      falling back to jiffies eventually. It's not as though
      random_get_entropy_fallback() is super high precision or guaranteed to
      be entropic, but basically anything that's not zero all the time is
      better than returning zero all the time.
      
      This is accomplished by just including the asm-generic code like on
      other architectures, which means we can get rid of the empty stub
      function here.
      
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Acked-by: default avatarMax Filippov <jcmvbkbc@gmail.com>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      9bfbcb37
    • Jason A. Donenfeld's avatar
      sparc: use fallback for random_get_entropy() instead of zero · fa15650b
      Jason A. Donenfeld authored
      commit ac9756c7
      
       upstream.
      
      In the event that random_get_entropy() can't access a cycle counter or
      similar, falling back to returning 0 is really not the best we can do.
      Instead, at least calling random_get_entropy_fallback() would be
      preferable, because that always needs to return _something_, even
      falling back to jiffies eventually. It's not as though
      random_get_entropy_fallback() is super high precision or guaranteed to
      be entropic, but basically anything that's not zero all the time is
      better than returning zero all the time.
      
      This is accomplished by just including the asm-generic code like on
      other architectures, which means we can get rid of the empty stub
      function here.
      
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: David S. Miller <davem@davemloft.net>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      fa15650b
    • Jason A. Donenfeld's avatar
      um: use fallback for random_get_entropy() instead of zero · 9dfc1459
      Jason A. Donenfeld authored
      commit 9f13fb0c
      
       upstream.
      
      In the event that random_get_entropy() can't access a cycle counter or
      similar, falling back to returning 0 is really not the best we can do.
      Instead, at least calling random_get_entropy_fallback() would be
      preferable, because that always needs to return _something_, even
      falling back to jiffies eventually. It's not as though
      random_get_entropy_fallback() is super high precision or guaranteed to
      be entropic, but basically anything that's not zero all the time is
      better than returning zero all the time.
      
      This is accomplished by just including the asm-generic code like on
      other architectures, which means we can get rid of the empty stub
      function here.
      
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Richard Weinberger <richard@nod.at>
      Cc: Anton Ivanov <anton.ivanov@cambridgegreys.com>
      Acked-by: default avatarJohannes Berg <johannes@sipsolutions.net>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      9dfc1459
    • Jason A. Donenfeld's avatar
      x86/tsc: Use fallback for random_get_entropy() instead of zero · 0cc41e2c
      Jason A. Donenfeld authored
      commit 3bd4abc0
      
       upstream.
      
      In the event that random_get_entropy() can't access a cycle counter or
      similar, falling back to returning 0 is suboptimal. Instead, fallback
      to calling random_get_entropy_fallback(), which isn't extremely high
      precision or guaranteed to be entropic, but is certainly better than
      returning zero all the time.
      
      If CONFIG_X86_TSC=n, then it's possible for the kernel to run on systems
      without RDTSC, such as 486 and certain 586, so the fallback code is only
      required for that case.
      
      As well, fix up both the new function and the get_cycles() function from
      which it was derived to use cpu_feature_enabled() rather than
      boot_cpu_has(), and use !IS_ENABLED() instead of #ifndef.
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Reviewed-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Borislav Petkov <bp@alien8.de>
      Cc: x86@kernel.org
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      0cc41e2c
    • Jason A. Donenfeld's avatar
      nios2: use fallback for random_get_entropy() instead of zero · f2a6e872
      Jason A. Donenfeld authored
      commit c04e7270
      
       upstream.
      
      In the event that random_get_entropy() can't access a cycle counter or
      similar, falling back to returning 0 is really not the best we can do.
      Instead, at least calling random_get_entropy_fallback() would be
      preferable, because that always needs to return _something_, even
      falling back to jiffies eventually. It's not as though
      random_get_entropy_fallback() is super high precision or guaranteed to
      be entropic, but basically anything that's not zero all the time is
      better than returning zero all the time.
      
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Acked-by: default avatarDinh Nguyen <dinguyen@kernel.org>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      f2a6e872
    • Jason A. Donenfeld's avatar
      arm: use fallback for random_get_entropy() instead of zero · db1d13fe
      Jason A. Donenfeld authored
      commit ff8a8f59
      
       upstream.
      
      In the event that random_get_entropy() can't access a cycle counter or
      similar, falling back to returning 0 is really not the best we can do.
      Instead, at least calling random_get_entropy_fallback() would be
      preferable, because that always needs to return _something_, even
      falling back to jiffies eventually. It's not as though
      random_get_entropy_fallback() is super high precision or guaranteed to
      be entropic, but basically anything that's not zero all the time is
      better than returning zero all the time.
      
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Reviewed-by: default avatarRussell King (Oracle) <rmk+kernel@armlinux.org.uk>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      db1d13fe
    • Jason A. Donenfeld's avatar
      mips: use fallback for random_get_entropy() instead of just c0 random · 6fa912f9
      Jason A. Donenfeld authored
      commit 1c99c6a7
      
       upstream.
      
      For situations in which we don't have a c0 counter register available,
      we've been falling back to reading the c0 "random" register, which is
      usually bounded by the amount of TLB entries and changes every other
      cycle or so. This means it wraps extremely often. We can do better by
      combining this fast-changing counter with a potentially slower-changing
      counter from random_get_entropy_fallback() in the more significant bits.
      This commit combines the two, taking into account that the changing bits
      are in a different bit position depending on the CPU model. In addition,
      we previously were falling back to 0 for ancient CPUs that Linux does
      not support anyway; remove that dead path entirely.
      
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Tested-by: default avatarMaciej W. Rozycki <macro@orcam.me.uk>
      Acked-by: default avatarThomas Bogendoerfer <tsbogend@alpha.franken.de>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      6fa912f9
    • Jason A. Donenfeld's avatar
      m68k: use fallback for random_get_entropy() instead of zero · 36f38f83
      Jason A. Donenfeld authored
      commit 0f392c95
      
       upstream.
      
      In the event that random_get_entropy() can't access a cycle counter or
      similar, falling back to returning 0 is really not the best we can do.
      Instead, at least calling random_get_entropy_fallback() would be
      preferable, because that always needs to return _something_, even
      falling back to jiffies eventually. It's not as though
      random_get_entropy_fallback() is super high precision or guaranteed to
      be entropic, but basically anything that's not zero all the time is
      better than returning zero all the time.
      
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Acked-by: default avatarGeert Uytterhoeven <geert@linux-m68k.org>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      36f38f83
    • Jason A. Donenfeld's avatar
      timekeeping: Add raw clock fallback for random_get_entropy() · a7d04ca9
      Jason A. Donenfeld authored
      commit 1366992e
      
       upstream.
      
      The addition of random_get_entropy_fallback() provides access to
      whichever time source has the highest frequency, which is useful for
      gathering entropy on platforms without available cycle counters. It's
      not necessarily as good as being able to quickly access a cycle counter
      that the CPU has, but it's still something, even when it falls back to
      being jiffies-based.
      
      In the event that a given arch does not define get_cycles(), falling
      back to the get_cycles() default implementation that returns 0 is really
      not the best we can do. Instead, at least calling
      random_get_entropy_fallback() would be preferable, because that always
      needs to return _something_, even falling back to jiffies eventually.
      It's not as though random_get_entropy_fallback() is super high precision
      or guaranteed to be entropic, but basically anything that's not zero all
      the time is better than returning zero all the time.
      
      Finally, since random_get_entropy_fallback() is used during extremely
      early boot when randomizing freelists in mm_init(), it can be called
      before timekeeping has been initialized. In that case there really is
      nothing we can do; jiffies hasn't even started ticking yet. So just give
      up and return 0.
      Suggested-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Reviewed-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Theodore Ts'o <tytso@mit.edu>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      a7d04ca9
    • Jason A. Donenfeld's avatar
      powerpc: define get_cycles macro for arch-override · eb2f9d72
      Jason A. Donenfeld authored
      commit 40883583
      
       upstream.
      
      PowerPC defines a get_cycles() function, but it does not do the usual
      `#define get_cycles get_cycles` dance, making it impossible for generic
      code to see if an arch-specific function was defined. While the
      get_cycles() ifdef is not currently used, the following timekeeping
      patch in this series will depend on the macro existing (or not existing)
      when defining random_get_entropy().
      
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Benjamin Herrenschmidt <benh@ozlabs.org>
      Cc: Paul Mackerras <paulus@samba.org>
      Acked-by: default avatarMichael Ellerman <mpe@ellerman.id.au>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      eb2f9d72
    • Jason A. Donenfeld's avatar
      alpha: define get_cycles macro for arch-override · 10455a36
      Jason A. Donenfeld authored
      commit 1097710b
      
       upstream.
      
      Alpha defines a get_cycles() function, but it does not do the usual
      `#define get_cycles get_cycles` dance, making it impossible for generic
      code to see if an arch-specific function was defined. While the
      get_cycles() ifdef is not currently used, the following timekeeping
      patch in this series will depend on the macro existing (or not existing)
      when defining random_get_entropy().
      
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Richard Henderson <rth@twiddle.net>
      Cc: Ivan Kokshaysky <ink@jurassic.park.msu.ru>
      Acked-by: default avatarMatt Turner <mattst88@gmail.com>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      10455a36
    • Jason A. Donenfeld's avatar
      parisc: define get_cycles macro for arch-override · 5f0b77ca
      Jason A. Donenfeld authored
      commit 8865bbe6
      
       upstream.
      
      PA-RISC defines a get_cycles() function, but it does not do the usual
      `#define get_cycles get_cycles` dance, making it impossible for generic
      code to see if an arch-specific function was defined. While the
      get_cycles() ifdef is not currently used, the following timekeeping
      patch in this series will depend on the macro existing (or not existing)
      when defining random_get_entropy().
      
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Acked-by: default avatarHelge Deller <deller@gmx.de>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      5f0b77ca
    • Jason A. Donenfeld's avatar
      s390: define get_cycles macro for arch-override · 80459abc
      Jason A. Donenfeld authored
      commit 2e3df523
      
       upstream.
      
      S390x defines a get_cycles() function, but it does not do the usual
      `#define get_cycles get_cycles` dance, making it impossible for generic
      code to see if an arch-specific function was defined. While the
      get_cycles() ifdef is not currently used, the following timekeeping
      patch in this series will depend on the macro existing (or not existing)
      when defining random_get_entropy().
      
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Cc: Vasily Gorbik <gor@linux.ibm.com>
      Cc: Alexander Gordeev <agordeev@linux.ibm.com>
      Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
      Cc: Sven Schnelle <svens@linux.ibm.com>
      Acked-by: default avatarHeiko Carstens <hca@linux.ibm.com>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      80459abc
    • Jason A. Donenfeld's avatar
      ia64: define get_cycles macro for arch-override · 73385644
      Jason A. Donenfeld authored
      commit 57c0900b
      
       upstream.
      
      Itanium defines a get_cycles() function, but it does not do the usual
      `#define get_cycles get_cycles` dance, making it impossible for generic
      code to see if an arch-specific function was defined. While the
      get_cycles() ifdef is not currently used, the following timekeeping
      patch in this series will depend on the macro existing (or not existing)
      when defining random_get_entropy().
      
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: Arnd Bergmann <arnd@arndb.de>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      73385644
    • Jason A. Donenfeld's avatar
      init: call time_init() before rand_initialize() · 5fac8666
      Jason A. Donenfeld authored
      commit fe222a6c
      
       upstream.
      
      Currently time_init() is called after rand_initialize(), but
      rand_initialize() makes use of the timer on various platforms, and
      sometimes this timer needs to be initialized by time_init() first. In
      order for random_get_entropy() to not return zero during early boot when
      it's potentially used as an entropy source, reverse the order of these
      two calls. The block doing random initialization was right before
      time_init() before, so changing the order shouldn't have any complicated
      effects.
      
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Reviewed-by: default avatarStafford Horne <shorne@gmail.com>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      5fac8666
    • Jason A. Donenfeld's avatar
      random: fix sysctl documentation nits · b88ae87b
      Jason A. Donenfeld authored
      commit 069c4ea6
      
       upstream.
      
      A semicolon was missing, and the almost-alphabetical-but-not ordering
      was confusing, so regroup these by category instead.
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      b88ae87b
    • Jason A. Donenfeld's avatar
      random: document crng_fast_key_erasure() destination possibility · 46542574
      Jason A. Donenfeld authored
      commit 8717627d upstream.
      
      This reverts 35a33ff3
      
       ("random: use memmove instead of memcpy for
      remaining 32 bytes"), which was made on a totally bogus basis. The thing
      it was worried about overlapping came from the stack, not from one of
      its arguments, as Eric pointed out.
      
      But the fact that this confusion even happened draws attention to the
      fact that it's a bit non-obvious that the random_data parameter can
      alias chacha_state, and in fact should do so when the caller can't rely
      on the stack being cleared in a timely manner. So this commit documents
      that.
      Reported-by: default avatarEric Biggers <ebiggers@kernel.org>
      Reviewed-by: default avatarEric Biggers <ebiggers@google.com>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      46542574
    • Jason A. Donenfeld's avatar
      random: make random_get_entropy() return an unsigned long · ec07b349
      Jason A. Donenfeld authored
      commit b0c3e796
      
       upstream.
      
      Some implementations were returning type `unsigned long`, while others
      that fell back to get_cycles() were implicitly returning a `cycles_t` or
      an untyped constant int literal. That makes for weird and confusing
      code, and basically all code in the kernel already handled it like it
      was an `unsigned long`. I recently tried to handle it as the largest
      type it could be, a `cycles_t`, but doing so doesn't really help with
      much.
      
      Instead let's just make random_get_entropy() return an unsigned long all
      the time. This also matches the commonly used `arch_get_random_long()`
      function, so now RDRAND and RDTSC return the same sized integer, which
      means one can fallback to the other more gracefully.
      
      Cc: Dominik Brodowski <linux@dominikbrodowski.net>
      Cc: Theodore Ts'o <tytso@mit.edu>
      Acked-by: default avatarThomas Gleixner <tglx@linutronix.de>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      ec07b349
    • Jason A. Donenfeld's avatar
      random: allow partial reads if later user copies fail · fe156368
      Jason A. Donenfeld authored
      commit 5209aed5
      
       upstream.
      
      Rather than failing entirely if a copy_to_user() fails at some point,
      instead we should return a partial read for the amount that succeeded
      prior, unless none succeeded at all, in which case we return -EFAULT as
      before.
      
      This makes it consistent with other reader interfaces. For example, the
      following snippet for /dev/zero outputs "4" followed by "1":
      
        int fd;
        void *x = mmap(NULL, 4096, PROT_WRITE, MAP_ANONYMOUS | MAP_PRIVATE, -1, 0);
        assert(x != MAP_FAILED);
        fd = open("/dev/zero", O_RDONLY);
        assert(fd >= 0);
        printf("%zd\n", read(fd, x, 4));
        printf("%zd\n", read(fd, x + 4095, 4));
        close(fd);
      
      This brings that same standard behavior to the various RNG reader
      interfaces.
      
      While we're at it, we can streamline the loop logic a little bit.
      Suggested-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      Cc: Jann Horn <jannh@google.com>
      Signed-off-by: default avatarJason A. Donenfeld <Jason@zx2c4.com>
      Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@linuxfoundation.org>
      fe156368