Skip to content

Match Gaussian blur kernels to floating image dtypes #146

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
satiricalguru wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Match Gaussian blur kernels to floating image dtypes #146

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions dm_pix/_src/augment.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -492,6 +492,7 @@ def gaussian_blur(
# DO NOT REMOVE - Logging usage.

chex.assert_rank(image, {3, 4})
chex.assert_type(image, float)
chex.assert_size(sigma, 1)
if isinstance(sigma, chex.Scalar):
chex.assert_scalar_non_negative(sigma)
Expand Down Expand Up @@ -533,9 +534,12 @@ def _gaussian_blur(
num_channels = image.shape[channel_axis]
radius = int(kernel_size / 2)
kernel_size_ = 2 * radius + 1
x = jnp.arange(-radius, radius + 1).astype(jnp.float32)
# Compute the weights in at least float32, then match the image dtype.
filter_dtype = jnp.result_type(image.dtype, jnp.float32)
x = jnp.arange(-radius, radius + 1, dtype=filter_dtype)
sigma = jnp.asarray(sigma, dtype=filter_dtype)
blur_filter = jnp.exp(-(x**2) / (2.0 * sigma**2))
blur_filter = blur_filter / jnp.sum(blur_filter)
blur_filter = (blur_filter / jnp.sum(blur_filter)).astype(image.dtype)

@mangeshraut712 mangeshraut712 Nov 10, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Casting the normalised filter to image.dtype fixes the dtype error, but for bf16 it re-quantises the weights to an 8-bit mantissa after normalisation. The kernel then no longer sums to 1. Both separable passes also run as bf16 convolutions, so the error compounds. With kernel_size=3 it stays within the 2 * eps the new test allows. With a wider kernel (e.g. kernel_size=15, sigma=4), I'd expect a constant image to drift slightly in brightness. I haven't measured that, so it's worth a quick check.

An alternative with the same output dtype: upcast the image to filter_dtype, keep the filter in filter_dtype, run both _depthwise_conv2d passes there, and cast blurred back to the input dtype at the end. That costs one extra cast, keeps the f16/bf16 result as close as possible to the f32 reference, and avoids depending on how each backend accumulates bf16 convolutions. Adding a larger-kernel case to test_gaussian_blur_low_precision_image would show whether it matters.

blur_v = jnp.reshape(blur_filter, [kernel_size_, 1, 1, 1])
blur_h = jnp.reshape(blur_filter, [1, kernel_size_, 1, 1])
blur_h = jnp.tile(blur_h, [1, 1, 1, num_channels])
Expand Down
13 changes: 13 additions & 0 deletions dm_pix/_src/augment_test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -792,6 +792,19 @@ def test_gaussian_blur_sigma_types(self):
result = augment.gaussian_blur(image, sigma=sigma, kernel_size=3)
np.testing.assert_allclose(result, expected, rtol=1e-6, atol=1e-6)

@parameterized.product(dtype=(jnp.float16, jnp.bfloat16), sigma=(0.0, 0.8))
def test_gaussian_blur_low_precision_image(self, dtype, sigma):
image = jnp.arange(20, dtype=dtype).reshape(4, 5, 1) / 32
blur = functools.partial(augment.gaussian_blur, kernel_size=3)
expected = blur(image.astype(jnp.float32), sigma).astype(dtype)
result = jax.jit(blur)(image, sigma)
self.assertEqual(result.dtype, image.dtype)
np.testing.assert_allclose(
result.astype(jnp.float32),
expected.astype(jnp.float32),
rtol=2 * jnp.finfo(dtype).eps,
)

def test_negative_scalar_sigma_for_gaussian_blur(self):
image = jnp.zeros((4, 4, 3))
with self.assertRaises(AssertionError):
Expand Down
Loading