Picking Sprites

[ZacharyKamerling]

Here you can see it's picking the bottom right card but my cursor is nowhere near it.

I have a MainCamera (layer 0) and a CardCamera (layer 1). This problem occurs when I move the MainCamera. It's correctly picking things on layer 0, but not layer 1.

#[derive(Component)]
pub struct CardCamera;

#[derive(Component)]
pub struct MainCamera;

pub fn setup(mut commands: Commands) {
    commands.spawn((
        Camera2dBundle::default(),
        RenderLayers::layer(0),
        MainCamera,
    ));
    commands.spawn((
        Camera2dBundle {
            camera_2d: Camera2d {
                clear_color: ClearColorConfig::None,
            },
            camera: Camera {
                order: 1,
                ..default()
            },
            ..default()
        },
        RenderLayers::layer(1),
        CardCamera,
    ));
}

Here I'm creating the card entities

let mut sprite_bundle = SpriteBundle {...};
sprite_bundle.transform = Transform::from_xyz(...);

commands
    .spawn((
        sprite_bundle,
        RenderLayers::layer(1),
        PickableBundle::default(),
        On::<Pointer<DragStart>>::send_event::<CardDragStart>(),
        On::<Pointer<Drag>>::send_event::<CardDrag>(),
        On::<Pointer<DragEnd>>::send_event::<CardDragEnd>(),
    ));

[ZacharyKamerling]

I was able to remedy the problem by spawning the cameras in reverse order. Now I'm confused why layer 0 still works.

[aevyrie]

This may be fixed on main. Previously, the camera without a renderlayer was assumed to intersect with all other render layers, when it should only intersect with render layer 0.

[ZacharyKamerling]

Unfortunately the problem persists with

bevy_mod_picking = { git = "https://github.com/aevyrie/bevy_mod_picking.git", branch = "main" }

It must have something to do with camera iteration order because I can still remedy the problem by swapping the spawn order.

[aevyrie]

I'm having trouble replicating this. Can you provide a minimal repro, maybe a modification of the sprite or render_layer examples in this repo?

[ZacharyKamerling]

I'm having trouble replicating the problem as well. I'll clone my game repo and rip out parts of it until the problem goes away.

[ZacharyKamerling]

Here is a minimal(ish) example. Right click to drag the camera. If you left click where the sprite used to be, it will still fire Hello events. Swapping the cameras spawn order fixes the problem. If you replace the SpriteBundle with a MaterialMesh2dBundle, everything works fine.

use bevy::{
    input::mouse::MouseMotion, prelude::*, render::view::RenderLayers, sprite::MaterialMesh2dBundle,
};
use bevy_mod_picking::prelude::*;

pub fn main() {
    let mut app = App::new();
    app.add_plugins((DefaultPlugins, DefaultPickingPlugins))
        .add_systems(Startup, setup)
        .add_systems(Update, (hello, drag_main_camera))
        .add_event::<Hello>();
    app.run();
}

#[derive(Component)]
pub struct MainCamera;

#[derive(Event)]
pub struct Hello;

fn setup(
    mut commands: Commands,
    mut meshes: ResMut<Assets<Mesh>>,
    mut materials: ResMut<Assets<ColorMaterial>>,
    assets: Res<AssetServer>,
) {
    commands.spawn((
        Camera2dBundle {
            transform: Transform::from_xyz(0., 0., 0.),
            ..default()
        },
        RenderLayers::layer(0),
    ));

    commands.spawn((
        Camera2dBundle {
            camera: Camera {
                clear_color: ClearColorConfig::None,
                order: 1,
                ..default()
            },
            ..default()
        },
        RenderLayers::layer(1),
        MainCamera,
    ));

    commands.spawn((
        SpriteBundle {
            transform: Transform::from_xyz(-200., 0., 0.),
            texture: assets.load("imgs/whatever_img_you_desire.png"),
            ..default()
        },
        On::<Pointer<Click>>::send_event::<Hello>(),
        RenderLayers::layer(1),
        PickableBundle::default(),
    ));

    commands.spawn((
        MaterialMesh2dBundle {
            mesh: meshes.add(Circle::new(50.0)).into(),
            material: materials.add(ColorMaterial::from(Color::BLUE)),
            transform: Transform::from_xyz(200., 0., 0.),
            ..default()
        },
        RenderLayers::layer(0),
        PickableBundle::default(),
    ));
}

impl From<ListenerInput<Pointer<Click>>> for Hello {
    fn from(_: ListenerInput<Pointer<Click>>) -> Self {
        Hello
    }
}

pub fn drag_main_camera(
    buttons: Res<ButtonInput<MouseButton>>,
    mut motion_evr: EventReader<MouseMotion>,
    mut camera: Query<&mut Transform, With<MainCamera>>,
) {
    for ev in motion_evr.read() {
        if buttons.pressed(MouseButton::Right) {
            camera.single_mut().translation += Vec3::new(-ev.delta.x, ev.delta.y, 0.0);
        }
    }
}

pub fn hello(mut reader: EventReader<Hello>) {
    reader.read().for_each(|_| println!("hello"));
}

[nosjojo]

I also ran into issues picking with SpriteBundle, possibly related to this issue. In my example, I was clicking on sprites and performing 90 deg rotations on them. After the rotation, the picker seems to see the sprite in a larger area than it should, as if the raycast was coming from the side. I did the same trick as this user, swapping for a MaterialMesh2dBundle, and the issue went away.

20240526-1949-11.5809562.mp4

a slightly trimmed down version of my original code:

use bevy::{prelude::*, render::camera::ScalingMode};
use bevy_mod_picking::debug::DebugPickingMode;
use bevy_mod_picking::events::{Click, Pointer};
use bevy_mod_picking::prelude::*;
use bevy_mod_picking::{DefaultPickingPlugins, PickableBundle};
use std::f32::consts::PI;
use std::fmt;

/// We will store the world position of the mouse cursor here.
#[derive(Resource, Default)]
struct MyWorldCoords(Vec2);

/// Used to help identify our main camera
#[derive(Component)]
struct MainCamera;

fn main() {
    App::new()
        .init_resource::<MyWorldCoords>()
        .add_plugins((DefaultPlugins, DefaultPickingPlugins))
        .insert_resource(DebugPickingMode::Normal)
        .add_systems(Startup, (setup.before(load_cards), load_cards))
        .run();
}

fn setup(mut commands: Commands) {
    let mut camera = Camera2dBundle::default();
    camera.transform = Transform::from_xyz(910., 380., 0.);
    camera.projection.scaling_mode = ScalingMode::AutoMax {
        max_width: 3800.,
        max_height: 2100.,
    };
    commands.spawn((camera, MainCamera));
}

#[derive(Clone, Copy, Debug, PartialEq)]
enum Suit {
    Hearts,
    Clubs,
    Spades,
    Diamonds,
}
impl fmt::Display for Suit {
    fn fmt(&self, f: &mut fmt::Formatter) -> fmt::Result {
        match *self {
            Suit::Clubs => write!(f, "Clubs"),
            Suit::Hearts => write!(f, "Hearts"),
            Suit::Diamonds => write!(f, "Diamonds"),
            Suit::Spades => write!(f, "Spades"),
        }
    }
}

#[derive(Component, Debug)]
struct Card;
#[derive(Component, Debug)]
struct CardBack;

fn load_cards(mut commands: Commands, asset_server: Res<AssetServer>) {
    let suits = Vec::from([Suit::Hearts, Suit::Clubs, Suit::Spades, Suit::Diamonds]);
    let numbers = vec![
        "2", "3", "4", "5", "6", "7", "8", "9", "10", "J", "Q", "K", "A",
    ];

    let back_path = "cards/cardBack_blue2.png";
    let back_sprite: Handle<Image> = asset_server.load(back_path);

    for (x, suit) in suits.iter().enumerate() {
        for (val, num) in numbers.iter().enumerate() {
            let tform = Transform::from_xyz(val as f32 * 140., x as f32 * 190., 1.);
            let player_back = AnimationPlayer::default();
            let player_front = AnimationPlayer::default();
            let back_id = commands
                .spawn((
                    PickableBundle::default(),
                    On::<Pointer<Click>>::run(flip_card),
                    player_back,
                    CardBack,
                    Card,
                    SpriteBundle {
                        sprite: Sprite { ..default() },
                        texture: back_sprite.clone(),

                        global_transform: GlobalTransform::default(),
                        visibility: Visibility::Hidden,
                        ..default()
                    },
                ))
                .id();

            let path = format!("cards/card{suit}{num}.png");
            let img = asset_server.load(path);

            let front_id = commands
                .spawn((
                    PickableBundle::default(),
                    On::<Pointer<Click>>::run(flip_card),
                    player_front,
                    Card,
                    Name::new("card"),
                    SpriteBundle {
                        sprite: Sprite { ..default() },
                        texture: img.clone(),
                        transform: tform,
                        global_transform: GlobalTransform::default(),
                        ..default()
                    },
                ))
                .id();

            commands.entity(front_id).add_child(back_id);
        }
    }
}

fn animation_card_flip_half() -> AnimationClip {
    let mut animation = AnimationClip::default();
    let card_name = Name::new("card");
    animation.add_curve_to_path(
        EntityPath {
            parts: vec![card_name],
        },
        VariableCurve {
            keyframe_timestamps: vec![0.0, 1.0],
            keyframes: Keyframes::Rotation(vec![
                Quat::IDENTITY,
                Quat::from_axis_angle(Vec3::Y, PI / 2.),
            ]),
            interpolation: Interpolation::Linear,
        },
    );
    return animation;
}

fn flip_card(
    mut query: Query<&mut AnimationPlayer>,
    clicked: ListenerMut<Pointer<Click>>,
    mut animations: ResMut<Assets<AnimationClip>>,
) {
    if let Ok(mut player) = query.get_mut(clicked.listener()) {
        player.play(animations.add(animation_card_flip_half()));
        _ = player.is_finished();
    }
}

assets used in the load as well as the rest of the source from this (like the cargo.toml) can be found here: https://github.com/nosjojo/match-game/tree/d6e0891f1711fa94797983478d575dc66e6606e5