Repository navigation
Conversation
- add development principles, language preferences, test-driven workflow, and versioning SOP - update .gitignore with spec kit directories - add initial README description and documentation links - create basic package and docs structure
- Create basic NN struct with generic type support for float32/float64 - Add New() constructor function for creating neural network instances - Implement perceptron example with training loop and dataset - Add timing functionality to measure training performance - Include placeholder methods for Train, Verify, WriteConfig, WriteWeights - Add sample dataset and training verification logic
…ives Implement a complete set of activation functions including ELISH, ELU, LINEAR, LEAKY_RELU, RELU, SELU, SIGMOID, SOFTMAX, SWISH, and TANH with their corresponding derivative functions. Each function supports parameterization where applicable and includes proper float32/float64 generic support. Includes comprehensive test coverage with unit tests for all functions and their derivatives, edge case handling, and benchmark tests for performance validation. Also adds documentation table with function descriptions and parameters.
…tion Implement 17 different loss functions including MSE, MAE, CCE, BCE, MAPE, MSLE, KLD, COSINE, POISSON, HINGE, SQ_HINGE, CAT_HINGE, LOG_COSH, HUBER, AVG, RMSE, and ARCTAN. Each function includes proper error handling and type constraints for float32 and float64. Add Loss and LossVector functions for single and vector inputs with comprehensive test coverage including edge cases and benchmarks.
…moid struct - introduce ActivationFunction interface with activation and derivative methods - create Sigmoid struct with configurable slope parameter - implement activation and derivative methods for Sigmoid - replace function-based approach with interface-based design - maintain backward compatibility while improving extensibility
…lementation - Replace float32/float64 union types with utils.Float constraint - Add proper type conversions in activation and derivative functions - Fix sigmoid derivative calculation to use correct formula - Move floating point type constraint to shared utils package - Update method signatures to use consistent type parameter approach
- Rename ActivationType to Type in activation package for consistency - Rename LossType to Type in loss package for consistency - Fix C++-style syntax errors in perceptron example to proper Go syntax - Uncomment imports and enable proper neural network configuration - Add basic NN structure with SetHiddenLayers and SetOutputLayer methods - Include comprehensive code analysis report documenting issues found
…ral network setup - Fix type conversion: lenOutput to int for array indexing - Add safety check to prevent division by zero in error calculation - Implement proper method chaining by returning receiver instead of new instances - Enhance logging with dedicated logger instance in NN struct - Simplify perceptron example configuration and remove commented code
… and activation functions - Add InputCell, HiddenCell, OutputCell, and BiasCell implementations - Implement CoreCell with activation function support (sigmoid, tanh, relu, linear) - Add Axon type for neuron connections with weight calculation - Include comprehensive tests for all cell types and functionality - Add detailed README documentation with examples and usage instructions - Implement neuron interfaces for forward/backward propagation - Add activation function derivatives for backpropagation
Rename CoreCell to core, HiddenCell to Hidden, InputCell to Input, and OutputCell to Output across all files. Update README documentation to reflect new type names and fix minor formatting issues.
Update cell types to use shorter names (Hidden, Output, Bias) instead of (HiddenCell, OutputCell, BiasCell). Change return types from values to pointers for consistency. Remove unused methods and update axon weight calculation to accept pointer parameter.
- Move HiddenLayer and OutputLayer types to separate config.go file - Create new network package with Bundle and Network types - Update NN struct to use network.Network instead of direct implementation - Rename and reorganize neuron-related types and interfaces - Update axon imports to use neuron package instead of nn - Add comments and documentation to clarify new structure
Migrate from standard log package to slog for better structured logging capabilities. Replace log.Println with slog.Info and log.Warn for consistent logging interface across the neural network package.
…r usage Set DEFAULT constant for activation and loss packages to provide default values. Replace instance logger with global Logger in NN struct and update initialization with default values for activation, loss, rate, and bias parameters.
- Add CalculateTotalLoss function to compute aggregate loss values - Update network package with new bundle structure and cell naming - Move propagation logic to separate module for better organization - Update neuron interfaces with clearer documentation - Fix minor formatting issues across multiple files
…mentation - Change CalculateTotalLoss to accept Type value instead of pointer - Add Bundle methods for adding cells and getting misses - Update Network struct to use Bundle instances instead of pointers - Implement loss calculation using bundle misses in propagation - Add comprehensive Bundle implementation with cell management
…lify function signatures - Change CalculateTotalLoss to accept []*T instead of []T for misses parameter - Update Loss function to remove variadic params and use utils.Float constraint - Remove unused utils import from propagation.go - Simplify poisson and hinge loss function calls by removing params...
Add forward propagation methods including calculateValues and calculateLoss Implement backward propagation with calculateMisses and calculateWeights Replace commented Rust implementation with Go equivalents Update imports to use cell package for neuron operations
…ation logic - Simplify struct field names by removing redundant prefixes - Update method signatures to use embedded struct fields - Refactor propagation methods to use new field access patterns - Add missing interface methods for value and miss setting - Update variable names from Value/Miss to value/miss for consistency - Modify core struct to use private fields with getter/setter methods - Update neuron interfaces to include SetValue and SetMiss methods - Refactor calculateMissForHidden to use new interface methods - Add debug print statement in bundle SetTargets method
…agement add go module configuration files for main project and perceptron example remove go.mod and go.sum from gitignore to properly track dependencies update .gitignore to remove redundant git and editor backup file entries
- Change Bundle type from []Axon[T] to []*Axon[T] for proper pointer handling - Update Hidden cell to use axon.Bundle[T] type instead of []*axon.Axon[T] - Remove activation and loss test files that are no longer needed - Update network bundle to use neuron.Nucleus for type assertion
- Move core.go, hidden.go from pkg/cell to pkg/neuron/cell - Update import paths in affected files - Rename cell package alias in network.go for clarity
- Modified the comment for the Float interface to simply "Float" - Removed the redundant description "Define constraints for floating point types"
- Add logger package in utils for global logging - Remove duplicate logger initialization in nn package - Refactor network package by fixing import paths and renaming types - Move propagation.go from nn to network package - Add stub files for Query, Train, and Verify functions in nn package
- Reorganize import statements for better readability - Implement interface assertions for neuron types - Add type definitions for network components - Update package name from 'level' to 'network' - Add JSON, YAML, TOML, and XML struct tags to network components - Initialize isTrain field in NN struct
- Renamed loop index variables in bundle.go and propagation.go - Updated variable names to improve code readability - Changed 'cell' to 'c', 'neuron' to 'n', 'axon' to 'a' in respective loops
…d dropout support
…kwardSeq + FD check
…ock JSON round-trip
…orm branch (TRANS-3/9)
- Add mask []bool field to Dropout[T]; ApplyMask stores retain mask - Add BackwardMask(upstream) — scales retained grads by 1/p, zeroes dropped - Wire Drop1/Drop2 BackwardMask into EncoderBlock backwardPostNorm/backwardPreNorm - Add TestDropoutBackwardMask + TestDropoutBackwardMaskInference (regularizer 83.1%) - Add TestEncoderBlock_DropoutZeroEquivalence + TestEncoderBlock_DropoutTrainingDiffers Closes T-18A08 (Phase β complete).
…ANS-4) - Create decoder.go: DecoderBlock[T] identical to EncoderBlock but causal=true MHA - Post-norm/pre-norm Forward + Backward; MarshalJSON/UnmarshalJSON; ApplyGradSGD - Compile-time Block[T]/Layer[T]/MaskedLayer[T] assertions - decoder_test.go: 11 tests including BackwardFD, JSON round-trip, Causal flag check Closes T-18A09.
- pkg/quantization/ design: Quantize[T], QuantizedNetwork, CalibrationRunner - Three-phase plan: α weight-only Dense/Conv → β full-int8 GEMM → γ attention projections - Full Invariant Compliance table mapping QUANT-1..10 to Go types - .qnn.json persistence schema + SideBySideEvaluator contract - Enables /magic-task for Phase 19 planning
Activate Phase 19 from backlog: l2-quantization-impl.md Stable v0.1.0 authored + Phase 18 T-18A01..A09 Done (both scoping conditions met). 14 atomic tasks (T-19A01..A11 + T01..T02 + Z01) across three delivery phases α/β/γ. PLAN v2.19.0 → v2.20.0; TASKS v2.15.0 → v2.16.0.
…/nn wiring (v0.16.0 RC) - pkg/layer/transformer/: TransformerConfig[T], Block[T] interface, internal ffn[T], EncoderBlock[T] (post/pre-norm, dropout, JSON), DecoderBlock[T] (causal), Stack[T] (N-block sequential chain, independent RNG per block TRANS-C8) - pkg/layer/norm/: LayerNorm.Backward + ApplyGradSGD + ForwardSeq/BackwardSeq - pkg/nn/: WithEncoderBlock/WithDecoderBlock/WithEncoderStack/WithDecoderStack options; applyConvBackward extended with transformer switch cases - Tests: 88.3% transformer coverage, 84.2% norm, 75.5% nn; FD < 1e-3, JSON round-trip bit-exact, TRANS-6 residual identity, depth-12 gradient-flow, copy-task stability - Bug fix: lazy cache allocation in EncoderBlock/DecoderBlock.Forward for compile-time shape inference (computeConvChainOutput runs before Init)
…e link + Phase 19 activation
Delivers weight-only (α), full-int8 (β), and attention projection (γ) post-training quantization for GoNN. New pkg/quantization/ package: Quantize[T], QuantizedNetwork[T], CalibrationRunner[T], Evaluate[T], Load[T]; .qnn.json persistence; 91.4% coverage.
- Remove explicit type arguments from New and MustNew calls - Update option functions to use type inference - Simplify code while maintaining functionality
…ization softmaxActivation computed exp(x)/(exp(x)+1), which overflows to NaN for float32 inputs above ~88. Switch to the stable form 1/(1+exp(-x)) for x≥0. Add TestSoftmaxActivationStability covering inputs up to ±1000. In examples/iris, add minMaxNorm to scale all four features to [0,1] before training. Without it, unnormalized iris values (0–8 range) combined with the newly-correct LR=0.01 (applied since Phase 6) caused output-layer saturation and NaN loss. TestE05Accuracy now reaches train≥0.90 / test≥0.85 reliably. Also fix explicit generic type arguments in test files and clean up redundant comments in pkg/quantization.
Drop explicit [T] / [float64] / [float32] type arguments that Go 1.26 improved inference can now deduce from typed call-site arguments or typed callback parameters, across: - pkg/layer/output.go - pkg/network/propagation.go - pkg/nn/compile.go - pkg/quantization/quantizer.go - pkg/network/weight_state_test.go - pkg/neuron/axon/axon_test.go - pkg/nn/meta_test.go - pkg/nn/nn_test.go - pkg/nn/weight_init_optimizer_test.go - pkg/optimizer/one_cycle_lr_test.go - pkg/optimizer/reduce_on_plateau_test.go - pkg/quantization/quantization_test.go Also fix numerically stable sigmoid in pkg/activation/softmax.go to prevent NaN overflow for large positive inputs (float32 > 88.72).
… args (infertypeargs)
Drop explicit [T] type arguments where Go 1.26 improved inference can
deduce T from typed call-site arguments (untyped float literals, typed
option/callback params). Preserve required type args where T cannot be
inferred: untyped-int zero arg, nil arg, float32 specialisations.
Files changed: pkg/optimizer/{optimizer,scheduler}_test.go,
pkg/dataset/{dataset_test,mnist}.go, pkg/layer/norm/norm_test.go,
pkg/layer/recurrent/{cell,recurrent_test}.go,
pkg/quantization/{quantization_test,quantizer}.go,
examples/{continuation,iris}/main.go
- Fix ignored errors and check for nil axons slice - Update fmt.Fprintf calls to handle error - Use _ = to ignore unnecessary function results - Close resources properly in defer statements - Simplify code and remove redundant type declarations - Update NN struct method calls to be consistent - Apply staticcheck suggestions and ignore false positives
- Remove outdated //nolint:staticcheck comments in internal/lintaimeta/linter.go and pkg/aimeta/lint.go - Remove unnecessary //nolint:errcheck comment in pkg/compute/gpu/opencl/opencl_test.go
…parseDir function - Implement custom parseDir function to handle build tags and parse .go files - Update LintDir function in internal/lintaimeta/linter.go to use new parseDir - Update Check function in pkg/aimeta/lint.go to use new parseDir - Remove deprecated parser.ParseDir calls
Updated the path to the agent configuration file in the SKILL.md documentation. Changed from 'agents.json' to 'scripts/agents.json' to reflect the correct location of the configuration file within the project structure.
…initialization - Introduce WithWeightInitSeed[T] option to pin the weight initialization seed - Update Config struct to include WeightInitSeed field - Modify compile function to use the configured WeightInitSeed - Update examples to demonstrate usage of the new option - Enhance test cases to cover reproducibility aspects
- Implement MNIST file download and decompression in both examples - Update README.md files with new quick start instructions and flag descriptions - Add .gitignore entries for downloaded MNIST files - Modify main.go files to include download functionality and handle -download flag
- Remove outdated references to design documents and specifications - Simplify and clarify example descriptions - Standardize test naming conventions - Remove unnecessary comments and code
- Fix historic bug in CalculateMisses: omitting activation derivative folding corrupted multi-layer gradients. Now fixed and fused output pairs skip the derivative for maximum correctness and speed. - Implement InferDense for concurrent inference without shared state mutation. - Rebasing weights now preserves existing ones across structural changes, avoiding prior learning loss on grow/shrink operations. - ApplyFlatWeights rejects length mismatches with an error, preventing partial updates that corrupt the network. - WithLearningRate now takes effect on each AndTrain call as intended. - SOFTMAX activation is correctly vectorized for performance. - Input and target data are validated for finiteness to prevent NaN/Inf poisoning of the network. These changes improve correctness, performance, and stability of the network's training and inference processes.
- Add Save/Load/Resume methods to NN struct for persistence - Implement FitDataset for training with streaming datasets - Pause/Resume/Stop APIs for concurrent training control - Thread-safe Query for concurrent inference - Expand package overview in README - Refactor load.go to use nn.Load under the hood - Add new MNIST CNN example - Update README with new features and examples
…of-arrays weight storage - Introduce `compute.DenseKernels` interface for accelerated linear algebra operations - Implement structure-of-arrays weight storage in `Network` - Add `SetBackend` and `KernelsActive` methods to manage compute backend - Update weight-related methods to support new storage layout - Refactor dense engine to use new backend interface
- Reorder struct fields to improve GC scan efficiency - Implement GC-scan-optimal field order in multiple layers and components - Update methods to use new field order where applicable - Remove redundant comments and update documentation These changes reduce the range of heap-allocated structs scanned by the garbage collector, improving GC performance across multiple components of the neural network library.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.