Training Forward Pass Controller
Forward pass of encrypted training on the standard layout, with encrypted (trainable) parameters. See Encrypted Training.
-
class FHEONTrainForwardPassController : public FHEONTrainSupportController, protected FHEONANNController
Forward pass of the training pipeline, standard layout (one channel of the batch per ciphertext).
Convolution, 1x1 shortcut, batch normalization, ReLU, average and global average pooling, fully connected layer and softmax with encrypted (trainable) parameters, one broadcast scalar per ciphertext; every primitive caches what the backward pass needs. With pretrainedFromCsv, parameters without a trained ciphertext yet are read from the PyTorch CSV exports. FHEONTrainStandardForwardPassController is the multithreaded, optimized version (same files and keys).
Subclassed by FHEONTrainStandardForwardPassController
Public Functions
-
inline FHEONTrainForwardPassController(CryptoContext<DCRTPoly> ctx)
-
void setContext(CryptoContext<DCRTPoly> &in_context)
-
inline void setNumSlots(int numSlots)
-
vector<Ctext> he_forward_convolution_optimized(vector<Ctext> &encryptedInputs, int batchSize, int inputWidth, int inputChannels, int outputChannels, int stride, string layerName = "", bool pretrainedFromCsv = false)
3x3 convolution (padding 1, stride 1 or 2) of the training forward pass, with ciphertext (trainable) weights.
Every input channel is replicated over the ring and rotated into its 9 taps; every output channel sums, over the input channels and taps, the products with its encrypted weights, then (stride 2) downsamples all images at once (downsample_with_multiple_channels) and adds its encrypted bias. The inputs are cached for the backward pass.
Parameter files (training_data_dir): layerName_weights_{out}_{in}_{tap} (one broadcast scalar per file, created with Kaiming init on first use) and layerName_bias_{out}. Cached inputs: layerName_channel_{in}.
- Parameters:
encryptedInputs – One ciphertext per input channel, each holding the batch of feature maps (batchSize x inputWidth x inputWidth, image after image).
batchSize – Number of images in every ciphertext.
inputWidth – Width (and height) of the input feature maps.
inputChannels – Number of input channels.
outputChannels – Number of output channels.
stride – 1 or 2.
layerName – Prefix of the parameter and cache files.
pretrainedFromCsv – If true, a weight or bias without a trained ciphertext yet is read from weights_folder + layerName + “_weight.csv” / “_bias.csv” and used as a plaintext (and stored as its first ciphertext for the backward pass).
- Returns:
One ciphertext per output channel, the batch packed image after image at the output width.
-
vector<Ctext> he_forward_convolution_shortcut_optimized(vector<Ctext> &encryptedInputs, int batchSize, int inputWidth, int inputChannels, int outputChannels, int stride, string layerName = "", bool pretrainedFromCsv = false)
1x1 (no spatial tap), stride-capable convolution — used for ResNet’s “shortcut”/downsample residual path.
Since a 1x1 kernel has no spatial neighbors, this is structurally identical to he_forward_linear_multiple_outputs’s per-position linear combination (output[oc] = sum_in(weight[oc][in]*input[in]) + bias[oc], one distinct scalar weight per (in,out) pair, broadcast over the whole vector) — just evaluated at full input resolution and, when stride>1, followed by the same downsample_with_multiple_channels + bias-after-downsample treatment he_forward_convolution_optimized uses for its strided 3x3 case (matches the already-correct inference-only FHEONANNStandardBatchController:: he_batch_convolution_shortcut_optimized’s ordering: dense combine -> downsample -> bias).
Caches its inputs under FC-style “_in_neuron_” naming (not “_channel_”, since backward_pass_convolution_shortcut’s dWeight formula is the FC-style sum(gradOutput*input), not a tap-rotation correlation).
-
vector<Ctext> he_forward_global_avgpool(vector<Ctext> &encryptedInputs, int batchSize, int dataWidth, int inputChannels, string layerName = "")
Global average pool: collapses each channel’s whole (batchSize x dataWidth x dataWidth) feature map down to one scalar per image, packed compactly (contiguous, one slot per image) — matches FHEONANNStandardBatchController::he_batch_globalpool’s construction (rotate the whole channel by spatialSize to bring each image’s block to the front, EvalSum that block, EvalMerge the per-image sums into a compact vector), simplified since this codebase’s batchSize (16) never needs he_batch_globalpool’s chunked-merge optimization (that only kicks in once batchSize exceeds its rotatePositions parameter, e.g. 32 there).
Caches inputs under plain “_channel_” naming (matches backward_pass_ global_avgpool’s expectations, and every other spatial-map forward function’s convention).
-
vector<Ctext> he_forward_relu(vector<Ctext> &encryptedInputs, vector<int> scaleValues, int inputChannels, int vectorSize, int polyDegree = 59, string layerName = "")
Apply a homomorphic ReLU activation using Chebyshev polynomial approximation.
This function evaluates a ReLU activation on encrypted inputs using a Chebyshev polynomial approximation over a bounded interval. Each input channel is optionally rescaled using a plaintext mask to control the output magnitude before applying the polynomial.
The ReLU function is approximated as: f(x) = 0 if x < 0 f(x) = scaleVal * x otherwise
where the approximation is evaluated over the interval [lowerBound, upperBound]. The computation is parallelized across input channels.
- Parameters:
encryptedInputs – Vector of ciphertexts representing encrypted inputs, sized [inputChannels].
scaleValues – Per-channel scaling factors applied before ReLU.
inputChannels – Number of input channels.
vectorSize – Number of valid slots per ciphertext.
polyDegree – Degree of the Chebyshev polynomial approximation.
layerName – Prefix of the cached inputs (layerName_channel_{i}), used by backward_pass_relu.
- Returns:
A vector of ciphertexts where each element contains the encrypted ReLU-activated output for the corresponding input channel.
-
vector<Ctext> he_forward_avgPool_optimized(vector<Ctext> &encryptedInputs, int batchSize, int inputWidth, int kernelWidth, int stride = 2, string layerName = "")
Perform optimized secure average pooling on batched encrypted feature maps.
This function computes average pooling across multiple input channels for batched data under homomorphic encryption. For each channel, it partitions the input feature map into local regions of size (kernelWidth × kernelWidth), sums the values in each region, and applies scaling to obtain the average. The stride length determines how pooling regions are traversed across the feature map.
The operation is applied independently to each input channel. Within each channel, ciphertext rotations and additions are used efficiently to align and sum pooling regions across batches, minimizing the number of homomorphic operations. The output is one ciphertext per channel, where each ciphertext encodes the pooled results for all batches.
Note
This implementation is optimized for FHE-based CNN/ANN architectures where reducing the number of ciphertext rotations and multiplications is critical for efficiency.
The function assumes ciphertext packing follows contiguous layout: [batch0: channel_i feature map][batch1: channel_i feature map]…
- Parameters:
encryptedInputs – Vector of ciphertexts, one per input channel, each encoding batched feature maps of size (batchSize × inputWidth × inputWidth).
batchSize – Number of batches encoded in the ciphertexts.
inputWidth – Width (and height) of the input feature maps (assumed square).
kernelWidth – Width (and height) of the pooling kernel (assumed square).
stridingLen – Stride length used to slide the pooling window across the input feature maps.
layerName – Prefix of the cached inputs (layerName_channel_{i}).
- Returns:
vector<Ctext> Vector of ciphertexts where each element corresponds to one pooled channel across all batches. Each output ciphertext encodes multiple pooled values per batch, arranged as: [batch0 pooled values][batch1 pooled values]…
-
vector<Ctext> he_forward_linear_multiple_outputs(vector<Ctext> &encryptedInput, int vectorSize, int inputSize, int outputSize, string layerName = "", bool pretrainedFromCsv = false)
Fully connected layer of the training forward pass, one ciphertext per neuron.
output[ou] = sum_in weight[ou][in] * input[in] + bias[ou], with one distinct encrypted weight per (output, input) pair broadcast over the vectorSize slots (the images of the batch). The inputs are cached for the backward pass.
Parameter files: layerName_out_neuron_{ou}_in_{in} and layerName_out_neuron_{ou}_bias. Cached inputs: layerName_in_neuron_{in}.
- Parameters:
encryptedInputs – One ciphertext per input neuron (one value per image in the first vectorSize slots).
vectorSize – Number of slots of every neuron (the batch size after the global average pooling).
inputSize – Number of input neurons.
outputSize – Number of output neurons.
layerName – Prefix of the parameter and cache files.
pretrainedFromCsv – If true, missing parameters are read from weights_folder + layerName + “_weight.csv” / “_bias.csv” (flat [out][in], nn.Linear layout) as plaintexts, and stored as ciphertexts.
- Returns:
One ciphertext per output neuron.
-
vector<Ctext> he_forward_softmax(vector<Ctext> &encryptedInputs, const vector<int> &scalingValues, int inputSize, int appxPolyDegree = 59)
Computing approximate softmax using a Taylor-approximated exponential and a Newton-Raphson-approximated reciprocal.
We use the standard L1 softmax formulation: exp(x_i) / sum(exp(x_j)), with exp(x) approximated by its degree-2 Taylor polynomial (1 + x + x^2/2).
The sum of exponentials is strictly positive but only loosely bounded (its true per-slot value can range anywhere up to the calibration-based worst-case upper bound
sumVal, derived fromscalingValues), so its reciprocal is computed via Newton-Raphson on 1/sqrt(x) (then squared to get 1/x) rather than a Chebyshev polynomial fit of 1/x directly: a Chebyshev fit needs a fixed, tight interval, and any interval wide enough to safely contain 1/x’s domain here sits at or straddles x=0 — 1/x’s pole — which blows up the fitted polynomial’s coefficients and corrupts decoding (this is what used to crash with “the approximation error is too
high”). Newton-Raphson only needs a same-order-of-magnitude starting guess and converges quadratically, so a single scalar init derived from sumVal/2 is enough for the whole batch. Same pattern already used for BatchNorm’s inverse-std (see newtonraphson_approximate_inverse_sqrt).
- Parameters:
encryptedInputs – Vector of ciphertexts representing encrypted logits.
inputSize – Size of the input (not used directly, but keeps signature aligned).
appxPolyDegree – Unused now (kept for call-site compatibility) — the reciprocal step uses a fixed, small-iteration-count Newton-Raphson instead of a polynomial degree.
scalingValues – Per-class bound on |logit| (e.g. read_batch_scaling_values): sets the worst-case sum of the exponentials, hence the start of the Newton reciprocal.
- Returns:
Vector of ciphertexts containing softmax probabilities.
-
inline FHEONTrainForwardPassController(CryptoContext<DCRTPoly> ctx)