Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEAT] Add bilinear resize to RawImage #1101

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

[FEAT] Add bilinear resize to RawImage #1101

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
d4cfacb
Fix that resize could be an none whole number causing issues.
BritishWerewolf Dec 15, 2024
e337367
Implement bilinear resampling of RawImage. Fallbacks to original Canv…
BritishWerewolf Dec 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
100 changes: 82 additions & 18 deletions src/utils/image.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -350,11 +350,11 @@ export class RawImage {
* @param {Object} options Additional options for resizing.
* @param {0|1|2|3|4|5|string} [options.resample] The resampling method to use.
* @returns {Promise<RawImage>} `this` to support chaining.
* @throws {Error} If the width or height is not a whole number.
*/
async resize(width, height, {
resample = 2,
} = {}) {

// Do nothing if the image already has the desired size
if (this.width === width && this.height === height) {
return this;
Expand All @@ -363,41 +363,105 @@ export class RawImage {
// Ensure resample method is a string
let resampleMethod = RESAMPLING_MAPPING[resample] ?? resample;

const nullish_width = isNullishDimension(width);
const nullish_height = isNullishDimension(height);
// Width and height must be whole numbers.
if (!(Number.isInteger(width) || nullish_width)) {
throw new Error(`Width must be an integer, but got ${width}`);
}
if (!(Number.isInteger(height) || nullish_height)) {
throw new Error(`Height must be an integer, but got ${height}`);
}

// Calculate width / height to maintain aspect ratio, in the event that
// the user passed a null value in.
// This allows users to pass in something like `resize(320, null)` to
// resize to 320 width, but maintain aspect ratio.
const nullish_width = isNullishDimension(width);
const nullish_height = isNullishDimension(height);
if (nullish_width && nullish_height) {
return this;
} else if (nullish_width) {
width = (height / this.height) * this.width;
width = Math.round((height / this.height) * this.width);
} else if (nullish_height) {
height = (width / this.width) * this.height;
height = Math.round((width / this.width) * this.height);
}

if (IS_BROWSER_OR_WEBWORKER) {
// TODO use `resample` in browser environment

// Store number of channels before resizing
const numChannels = this.channels;

// Create canvas object for this image
const canvas = this.toCanvas();
// Create the output array for the resized image
const resizedData = new Uint8ClampedArray(width * height * numChannels);

// Scale factors for mapping new dimensions back to original dimensions
const xScale = this.width / width;
const yScale = this.height / height;

// Actually perform resizing using the canvas API
const ctx = createCanvasFunction(width, height).getContext('2d');
switch (resampleMethod) {
case 'bilinear':
// Iterate over each pixel in the new image.
for (let y = 0; y < height; y++) {
for (let x = 0; x < width; x++) {
// Map new coordinates to original coordinates.
const srcX = x * xScale;
const srcY = y * yScale;

// Calculate the surrounding pixels.
// Ensure that the pixels are within the bounds
// of the image.
const x0 = Math.floor(srcX);
const x1 = Math.min(x0 + 1, this.width - 1);
const y0 = Math.floor(srcY);
const y1 = Math.min(y0 + 1, this.height - 1);

// Calculate fractional parts for interpolation.
const dx = srcX - x0;
const dy = srcY - y0;

for (let c = 0; c < numChannels; c++) {
// Get the values of the new pixel area.
// Always multiply by the width because we
// storing the data in a 1D array.
// To get the second row, we must add a full
// width, then adding the x offset.
const topLeft = this.data[(((y0 * this.width) + x0) * numChannels) + c];
const topRight = this.data[(((y0 * this.width) + x1) * numChannels) + c];
const bottomLeft = this.data[(((y1 * this.width) + x0) * numChannels) + c];
const bottomRight = this.data[(((y1 * this.width) + x1) * numChannels) + c];
Comment on lines +421 to +429
Copy link
Contributor Author

@BritishWerewolf BritishWerewolf Dec 17, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if my comment makes sense, but basically if we had an area like this:

array
1 2 3 4 5 6 7 8 9 10 11 12

image
1  2  3  4
5  6  7  8
9 10 11 12

To get the element on the third row (3 - 1 since it is zero indexed) and second column (2 - 1), we must multiply by the width (4), then add the column.
In this case, we would do:

((row - 1) * width) + (column - 1) = index
((3 - 1) * 4) + (2 - 1) = 9


// Perform bilinear interpolation.
// Find the horizontal position along the
// top and bottom rows.
const top = (topLeft * (1 - dx)) + (topRight * dx);
const bottom = (bottomLeft * (1 - dx)) + (bottomRight * dx);
// Find the value between these two values.
const interpolatedValue = (top * (1 - dy)) + (bottom * dy);
Comment on lines +431 to +437
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here, we need to get the x value for the top and bottom of the new resized area, then interpolate between these two values to find what the new y value.

topLeft          topRight
+--------T--------------+
|                       |
|                       |
|          I            |
|                       |
|                       |
+------------B----------+
bottomLeft    bottomRight

Here, top is represented by T, bottom is represented by B, and interpolatedValue is represented by I.


// Set the value in the resized data.
resizedData[(((y * width) + x) * numChannels) + c] = Math.round(interpolatedValue);
}
}
}
break;

// Draw image to context, resizing in the process
ctx.drawImage(canvas, 0, 0, width, height);
// Fallback to the Canvas API.
default:
// Create canvas object for this image
const canvas = this.toCanvas();

// Create image from the resized data
const resizedImage = new RawImage(ctx.getImageData(0, 0, width, height).data, width, height, 4);
// Actually perform resizing using the canvas API
const ctx = createCanvasFunction(width, height).getContext('2d');

// Convert back so that image has the same number of channels as before
return resizedImage.convert(numChannels);
// Draw image to context, resizing in the process
ctx.drawImage(canvas, 0, 0, width, height);

// Create image from the resized data
const resizedImage = new RawImage(ctx.getImageData(0, 0, width, height).data, width, height, 4);

// Convert back so that image has the same number of channels as before
return resizedImage.convert(numChannels);
}

return new RawImage(resizedData, width, height, numChannels);
} else {
// Create sharp image from raw data, and resize
let img = this.toSharp();
Expand Down Expand Up @@ -699,7 +763,7 @@ export class RawImage {
/**
* Split this image into individual bands. This method returns an array of individual image bands from an image.
* For example, splitting an "RGB" image creates three new images each containing a copy of one of the original bands (red, green, blue).
*
*
* Inspired by PIL's `Image.split()` [function](https://pillow.readthedocs.io/en/latest/reference/Image.html#PIL.Image.Image.split).
* @returns {RawImage[]} An array containing bands.
*/
Expand Down