If I use $x * $factorial($x - 1)

True, it should have been multiplication; thanks.

I get an out-of-range error as of $factorial(21).

Integers in BaseX are limited to 64bit (that’s something the spec allows). You can use decimals to work with greater numbers:

declare variable $factorial := fn($x) {
  if($x > 1) then $x * $factorial($x - 1) else $x
};
$factorial(1000.0)

If you need even larger results, you can use a tail-call-optimized variant…

declare variable $factorial := fn($x, $result) {
  if($x > 1) then $factorial($x - 1, $x * $result) else $result
};
$factorial(100000.0, 1)

…or fold-left:

fold-left(1 to 100000, 1.0, op('*'))

Hope this helps,
Christian