奥迪车连续3年未年检被强制报废 司机后悔不已

Non-prehensile manipulation, such as pivoting, pushing, and sliding, plays an important role in enhancing the dexterity of robotic systems [1, 2, 3]. These skills allow robots to interact with the environment more flexibly, enabling them to adapt to a wide range of tasks without requiring secure grasps. However, achieving such skills is challenging due to the inherently complex contact interactions (e.g., making-breaking contact, sliding-sticking contact). These interactions introduce non-smooth dynamics that are difficult to model and control as the number of contacts increases.

Model-based optimization methods, such as CITO and Model Predictive Control (MPC) [4, 5, 6, 7, 8, 9], have demonstrated impressive performance, particularly in generating diverse trajectories at low computational cost. However, since these methods, in general, rely on simplified models of manipulation, they can be highly sensitive to uncertainties due to model inaccuracies. More critically, they often rely on offline system identification or online estimation of privileged information, such as object properties or contact states. This dependency limits the applicability of model-based controllers, particularly in real-world scenarios involving novel objects or partially observable environments.

Learning-based methods, such as RL, have also shown impressive performance, especially in their robustness against various sources of uncertainty [10, 11, 12, 13, 14, 15, 16]. These methods can operate without privileged information by directly learning policies from raw observations. However, they typically require a large number of training samples, resulting in long training times, which poses a significant challenge for practical deployment. This is especially problematic in non-prehensile manipulation, where the policy must reason object pose, contact locations, contact forces, and feasible action spaces from indirect and partial observations. Unlike prehensile manipulation (e.g., grasping [2]), where grasping provides stable control, non-prehensile tasks often involve underactuated dynamics and complex contact constraints that make the learning problem significantly harder. As a result, RL may often fail to discover viable solutions within a reasonable training time.

In this paper, we propose a framework that integrates the strengths of model-based planning with learning-based policy execution for non-prehensile pivoting manipulation. Our approach employs a student-teacher paradigm [17], as illustrated in Fig.?1. First, we employ CITO to collect a large number of task demonstrations across a range of privileged information parameters. Second, a teacher policy is trained in a high-fidelity simulator using RL, leveraging the demonstrations (e.g., robot, object, & contact trajectories) generated by CITO. By utilizing these demonstrations, the teacher policy achieves significantly higher sample efficiency than standard RL methods. Third, we train a student estimator to predict the privileged information required by the teacher policy. During training, the student estimator takes as input the history of sensor observations and segmentation features extracted from the vision pipeline, enabling it to infer the privileged information. Finally, we evaluate the trained policy in both simulation and hardware experiments, achieving zero-shot sim-to-real transfer. We verify that our framework substantially improves training efficiency compared to existing baselines. Moreover, we verify that our framework outperforms an MPC baseline, which struggles due to inaccuracies in privileged information. Our contributions are as follows.

• ?

  We propose a framework for learning contact-rich non-prehensile manipulation controllers and estimators by leveraging demonstrations generated by CITO.

• ?

  We develop a sim-to-real transfer approach based on a student-teacher architecture, where the student estimates privileged information from partial observations using a temporal history of visual and force sensing.

• ?

  We demonstrate that our method achieves robust manipulation performance against various uncertainties (e.g., object physical parameters) in real-world experiments.

Model-Based Optimization for Contact-Rich Manipulation. Model-based optimization methods have successfully achieved various non-prehensile manipulation skills, such as pushing [7, 5, 18], pivoting [19, 20, 21], and pulling [22, 23]. These methods design manipulation skills computationally efficiently by leveraging techniques such as contact smoothing [5, 24], mixed-integer convex optimization [19, 25], and distributed optimization [18, 26]. However, these methods typically require privileged information (i.e., full-state feedback). For example, Aydinoglu et?al. [27] assumes that contact forces in extrinsic contacts between the object and the environment are directly measurable, which becomes increasingly impractical as task complexity grows. In this paper, we relax the full-state feedback assumptions by adopting an RL approach, while still leveraging CITO to generate a large number of demonstrations. This strategy enables the agent to learn manipulation skills significantly more efficiently than standard RL methods that rely solely on sparse rewards.

Learning-Based Methods for Contact-Rich Manipulation. Learning-based methods, such as RL, Imitation Learning (IL), and foundation model-based methods, have demonstrated remarkable success in robotic manipulation [28, 29, 30, 31, 32, 33, 34, 35, 36, 37], enabling complex tasks such as bimanual cable manipulation and folding laundry. However, all of these methods require a large number of training samples, resulting in prohibitively long training times.

To improve sample efficiency, demonstration-guided RL has been studied [38, 11, 39], where the demonstrations are used to guide exploration of RL agent to learn the policy and improve sample efficiency. For example, Ota et?al. [40] uses Rapidly-exploring Random Trees (RRT) and Xiong et?al. [41] uses human videos for generating kinematically feasible demonstrations for manipulation. However, these works [40, 41, 42] only consider kinematically feasible demonstrations. Incorporating contact force information into demonstrations could be critical to learn fine manipulation due to very thin margins of error imposed by contact constraints. Although some works have explored dynamically feasible demonstrations in locomotion tasks [43, 44, 45], there has been relatively little work on applying such demonstrations to manipulation tasks. This is due to the lack of a reliable module for generating dynamically feasible demonstrations considering extrinsic contact states in manipulation. Some works [46, 47] leverage human demonstrations to capture contact forces, but collecting such data at scale is challenging and often requires significant manual effort. In contrast, we use CITO to automatically generate robot, object, and contact force trajectories, providing richer supervision and greater scalability than human demonstrations.

Bridging the sim-to-real gap is another key challenge. Privileged information used during training is often unavailable during real-world deployment. Some prior works reconstruct privileged states using external sensors [44, 45], such as AprilTags [48, 49]. The recent advances in the student-teacher framework [17, 50, 29, 51, 52, 53, 54, 55] enable zero-shot sim-to-real transfer by learning to predict privileged information. Although some works have applied the student-teacher framework to manipulation, they often rely on restrictive assumptions—for example, assuming that object size remains constant [56, 49]. In contrast, although we also adopt a student-teacher framework, we do not rely on such assumptions. By using a temporal history of force measurements and segmentation images, our student estimator is more broadly applicable to real-world scenarios involving novel objects.

In this section, we present our proposed framework, as shown in Fig.?1. The objective is to learn pivoting manipulation using only proprioceptive, visual, and force sensing. The proposed framework consists of three steps. In Step 1, task demonstrations are generated using CITO. In Step 2, a teacher policy which has access to the privileged information is trained using RL with the sampled demonstrations collected in Step 1. In Step 3, a student estimator is trained to estimate the privileged information, which serves as input to the teacher policy. The teacher policy with the predictions of the trained student estimator is ultimately deployed on physical hardware for real-world validation.

In this work, we make the following assumptions: (1) both the objects and the robots are rigid and the center of mass is located at the geometric centers, (2) manipulation occurs under quasi-static condition in SE(2), and (3) the robot end-effector pose, camera sensing, and robot contact force measurements are consistently available throughout manipulation.

Step 1: Collecting Demonstrations Using Contact-Implicit Trajectory Optimization We collect a large set of datasets using CITO in [57]. For $N_{r}$ robots, we consider the following CITO:

where $\bar{\mathbf{q}}_{t}:=[\bar{\mathbf{q}}_{t}^{o},\bar{\mathbf{q}}_{t}^{r}]$ and $\bar{\mathbf{y}}_{t}:=[\bar{\boldsymbol{\lambda}}_{t}^{e},\bar{\boldsymbol{\lambda}}_{t}^{r},\bar{\mathbf{z}}_{t}]$. $\bar{\mathbf{q}}_{t}^{o}\in\mathbb{R}^{3}$ represent an object pose in SE(2) and $\bar{\mathbf{q}}_{t}^{r}\in\mathbb{R}^{2\times N_{r}}$ represent robot end-effector positions in SE(2), respectively. The end-effector orientation is kept fixed throughout the task. $\bar{\boldsymbol{\lambda}}_{t}^{e}\in\mathbb{R}^{2\times N_{e}}$ and $\bar{\boldsymbol{\lambda}}_{t}^{r}\in\mathbb{R}^{2\times N_{r}}$ represent contact forces between an object and the environment, and between an object and the robots, respectively. $N_{e}$ represents the potential extrinsic number of contacts between the object and the environment. We denote $\bar{\mathbf{z}}_{t}\in\mathbb{R}^{2\times N_{e}}$ as the extrinsic contact location between the object and the environment. $\bar{\mathbf{q}}_{t}^{\text{ref}}\in\mathbb{R}^{3}$ represents the linear interpolation between the start and goal object pose with $T$ steps. We use the subscript $t$ to represent the timestep $t$. We denote $f_{\text{dyn}}$ as non-smooth dynamics of the non-prehensile manipulation, including nonsmooth contact switching, force and moment balance, and friction cone constraints. We denote $g$ as non-dynamics related constraints, such as bounds of decision variables and collision-avoidance. We emphasize that the generation of trajectories that satisfy kinematic feasibility alone and not dynamic feasibility are simple to obtain by removing some of the $f_{\text{dyn}}$ constraints, such as force and moment balance constraints. Thus, we denote kinematically feasible dynamics as $f_{\text{kin}}$. The problem (1) is solved using solvers such as Gurobi [58] and SNOPT [59]. See [57] and the appendix for more details. Solving (1) generates $N$ demonstrations $D_{\text{TO}}:=\{D_{\text{TO}}^{i}\}_{i=1}^{N}$, where $D_{\text{TO}}^{i}:=\{\{\bar{\mathbf{q}}_{t}\}_{t=0}^{T},\{\bar{\mathbf{y}}_{t}\}_{t=0}^{T}\}^{i}$. While previous works (e.g., [40, 41, 42]) only consider $\bar{\mathbf{q}}_{t}$ with $f_{\text{kin}}$, this work explicitly considers $\bar{\mathbf{q}}_{t}$ and $\bar{\mathbf{y}}_{t}$ with $f_{\text{dyn}}$. In particular, $\bar{\boldsymbol{\lambda}}_{t}^{r}$ guides for agents to learn robot motion direction, while $\bar{\boldsymbol{\lambda}}_{t}^{e}$ and $\bar{\mathbf{z}}_{t}$ offer insights into preferred extrinsic contacts.

Step 2: Learning Privileged Teacher-Policy In this step, a teacher policy is trained to achieve the desired pivoting manipulation in a simulation where privileged information is accessible. We formulate the problem as a Markov Decision Process (MDP), with each component defined as follows.

States. States consist of the privileged and non-privileged information. The privileged information $\mathbf{p}_{t}$ includes the object pose ${\mathbf{q}}_{t}^{o}\in\mathbb{R}^{3}$, the object and environment properties $\mathbf{v}_{t}\in\mathbb{R}^{N_{p}}$, and the extrinsic contact signal $\mathbf{b}_{t}\in\mathbb{Z}^{N_{e}}$. The object pose ${\mathbf{q}}_{t}^{o}$ lies in the SE(2) and consists of two positional components and one orientation. $\mathbf{v}_{t}$ encodes physical properties, which are the mass and size of the object, and the friction constants of both the object and the surrounding environment geometry. The extrinsic contact signal $\mathbf{b}_{t}$ is a binary vector where each element indicates whether a specific face of the object is in contact with a predefined environment surface (e.g., wall, table).

The non-privileged information $\mathbf{o}_{t}$ consists of the robot positions ${\mathbf{q}}_{t}^{r}\in\mathbb{R}^{2\times{N_{r}}}$, the binary robot contact signal ${d}_{t}\in\mathbb{Z}^{1}$, and the 2D contact forces ${\boldsymbol{\lambda}}_{t}^{r}\in\mathbb{R}^{2\times N_{r}}$ measured by force sensors mounted on the robots’ wrists. All observations are approximately normalized to lie in the range $[-1,1]$.

Actions. We consider linear translational actions in SE(2) for each robot, denoted as $\mathbf{a}_{t}^{2\times{N_{r}}}$. Specifically, each action represents a relative position command for the robots’ end-effector. These action commands are converted into joint torques using Operational Space Control (OSC) [60].

Rewards. Based on how demonstrations are used, we consider three distinct reward formulations. We denote three different RL polices using different demonstrations (i.e., using different reward formulation) as (1) Vanilla RL, which does not use any demonstrations, (2) Kinematics-conditioned RL, and (3) Dynamics-conditioned RL. These policies are obtained by 3 different rewards defined?as:

First, the progress reward is $r_{p}=\alpha_{1}\left(\frac{\pi}{2}-\theta_{e}\right)+\alpha_{2}\left(\theta_{e}^{2}\right)$, where $\theta_{e}=\arccos{\left(\frac{1}{2}\left(Tr\left(R^{\text{G}}R\right)-1\right)\right)}$. $Tr(\cdot)$ denotes the matrix trace, and $R$ and $R^{\text{G}}$ are the goal and current rotation matrices, respectively. $\theta_{e}$ measures the angular deviation between the current and goal orientations, and $\frac{\pi}{2}$ is added as the offset. While the linear term in $r_{p}$ is used in [61, 62], our experiments reveal that the inclusion of the quadratic term is necessary to achieve higher success rates under domain randomization (DR)?[63] over the size of the objects, which was not discussed in [62]. Second, the sparse success reward is defined as $r_{s}=\alpha_{3}\mathbb{I}_{G}\left(\mathbf{q}_{t}^{o}\right)$, where $\mathbb{I}_{G}$ is the indicator function over the goal set $G:=\left\{\mathbf{q}_{t}^{o}\in\mathbb{R}^{3}\mid\|\mathbf{q}_{t}^{o}-\mathbf{q}_{\text{goal}}^{o}\|\leq\epsilon_{s}\right\}$, where $\mathbf{q}_{\text{goal}}^{o}$ is the desired goal state of the object and $\epsilon_{s}$ is the user-specified positive constant. Third, the action smoothness reward is given by $r_{a}=\alpha_{4}\|{\mathbf{a}}_{t-1}-{\mathbf{a}}_{t}\|^{2}$, for avoiding non-smooth actions.

Next, we define the reward based on demonstrations generated by CITO. For the kinematic reward $r_{\text{kin}}$, we use object and robot poses $\bar{\mathbf{q}}_{t}$ and extrinsic contact locations $\bar{\mathbf{z}}_{t}$ obtained by solving (1) with $f_{\text{kin}}$. Note that contact force demonstrations are not available in this setting, as $f_{\text{kin}}$ does not have dynamics constraints. Thus, we compute $r_{\text{kin}}$ as:

where $\phi$ retrieves the closest reference robot configuration $\bar{\mathbf{q}}_{t}^{r}$ corresponding to the current object observation $\mathbf{q}_{t}^{o}$. Since both the object and environment parameters are sampled from a known dataset $D_{\text{TO}}$ during simulation, the corresponding object reference trajectory $\bar{\mathbf{q}}_{t}^{o}$ is known. Using the current observation, we identify the closest object configuration within this trajectory, and consequently retrieve the closest robot configuration. This reward term encourages the robot to follow the kinematically feasible behaviors.

Similarly, we define the dynamics reward $r_{\text{dyn}}$ by utilizing the demonstration $\bar{\mathbf{q}}_{t}$ and $\bar{\mathbf{y}}_{t}$ obtained by solving (1) with the dynamics model $f_{\text{dyn}}$:

where $\psi$ retrieves the closest reference robot contact forces $\bar{\boldsymbol{\lambda}}_{t}^{r}$ corresponding to $\mathbf{q}_{t}^{o}$, following the same logic as $\phi$. This reward encourages the robot to follow the dynamically feasible behaviors. In particular, the arccosine term in $r_{\text{dyn}}$ encourages the robot to perform a similar contact force direction as the demonstration shows. Importantly, we do not enforce matching the magnitude of the contact force, as we observe significant discrepancies between the dynamics model by $f_{\text{dyn}}$ and those in simulators (e.g., MuJoCo), leading to a potential sim2sim gap in contact force magnitudes. Hence, this work focuses on the direction of contact forces. The term $\mathbf{b}_{t}$ is used to count if the desired extrinsic contact states occur. The constants $\alpha_{i=1,2,3,8}$ are positive and the others are negative.

Step 3: Learning Student-Estimator The objective of this step is to train the student estimator only using sensor observations to predict the privileged information as shown in Fig.?1. We empirically observe that sensor observations alone are sufficient for the object whose geometry is in-distribution with the dataset. However, their reliability declines when there is uncertainty in object size, which is quite common when manipulating novel objects. To address this, we additionally incorporate vision inputs to improve the estimation of the privileged information. Directly using RGB images is avoided due to potential noise, and employing 3D point clouds is excluded due to their significant computational cost (see [29]). Instead, we leverage the object segmentation $\mathbf{s}_{t}$ derived from the RGB image, providing a compact but informative representation of the object.

Therefore, we define a student encoder that takes the history of the sensor observations, $[\mathbf{o}_{t-T},\cdots,\mathbf{o}_{t}]$, and the history of the segmentation features, $[\mathbf{s}_{t-T},\cdots,\mathbf{s}_{t}]$. Since $\mathbf{s}_{t}$ is high-dimensional, we first apply a Convolutional Neural Network (CNN) to compress the segmentation into a lower-dimensional feature representation $\mathbf{c}_{t}$. Using the temporal histories of $\mathbf{o}_{t}$ and $\mathbf{c}_{t}$, we use a Temporal Convolutional Network (TCN)?[64] to estimate the privileged information. We train CNN and TCN jointly via supervised learning using datasets collected by rolling out the teacher policy in the simulator under domain randomization. The supervised learning objective is to minimize the following loss function:

where $\mathbf{p}_{t}$ is the ground-truth privileged information and $\tilde{\mathbf{p}}_{t}$ is the estimated output from the student encoder. It is worth noting that we do not initialize the history buffer with zeros at the beginning of the episode as other works do (e.g., [56, 65, 66]). Instead, we populate the buffer by repeating the initial observation and include this initialization scheme in the supervised learning dataset, which was critical for the student estimator to achieve accurate performance.

We validate our framework across two distinct tasks (see Fig.?4): Pivoting with Wall: pivoting a box using an external wall, Pivoting without Wall: pivoting a box without relying on external support. For the latter task, the table surface must provide very high friction. In simulation, increasing the friction coefficient alone was insufficient to replicate the real-world behavior. As a workaround, we add a thin virtual wall of height $1\text{\,}\mathrm{mm}$ to simulate the effect of high-friction contact (see Fig.?4(b)). We define a trial as successful if the final orientation error satisfies $|\theta_{e}|\leq 0.087$ rad (i.e., $5\text{\,}\mathrm{\SIUnitSymbolDegree}$). We describe the setup for each module below, with additional details provided in the appendix.

Demonstration Setup. We use the method proposed in [57], randomizing object and environment parameters to generate diverse demonstrations. For all tasks, we randomize the mass of the object, the friction constant of the object and the environment, and the size of the object. For each task, we collect $5000$ demonstrations, which can be computed within a few minutes using $30$ Intel i9-13900K CPU cores.

Teacher Policy Setup. We train the teacher policy in MuJoCo simulator [67] using robosuite [68] as a wrapper. The agent is trained using Soft Actor Critic (SAC) [69], implemented with tf2rl [70]. For SAC, we use Multi-Layer Perceptron (MLP) for both actor and critic networks. The simulation runs at 500 Hz, while the policy operates at 10 Hz. For each episode, we set the maximum episode length to $300$ steps. Overall, training converges within 4 hours on a single NVIDIA RTX 4090. During training, we apply domain randomization over the objects’ mass and size, the friction constants of both the object and the environment, and the controller gains used in OSC within robosuite. Furthermore, we introduce sensor noise to both privileged information and sensor observations to account for the estimation errors from the student estimator during deployment.

Student Estimator Setup. We first rollout the trained teacher policy over $2000$ episodes and collect a dataset containing ground-truth privileged information, sensor observations, and corresponding segmentation images (640×480 resolution) of the object using MuJoCo’s rendering functionality. During data collection, we augment the segmentation images by introducing noise, such as randomly flipping, translating, and rotating segmentation masks, to improve robustness. We then train the student estimator via behavior cloning, minimizing the loss function (5) over multiple epochs. We use $T=5$ step history of the observations for training corresponding to 0.5 second. Overall, training converges with 10 epochs (1 hour roughly), depending on the range of domain randomization.

Hardware Setup. We use a 6 DoF MELFA robot [71] equipped with a stiffness controller and a 6-axis force/torque sensor. This hardware enables users to get robot end-effector positions and the force measurements in the world frame. For object segmentation, we use FastSAM [72] to generate multiple instance segmentations from an RGB image captured by an Intel RealSense D435 RGB-D camera [73]. To identify the target object, we filter the segmented instances under their corresponding point cloud information, under the assumption that a rough estimate of the SE(2) plane is available, as we focus exclusively on SE(2) planar manipulation.

Baselines. We implement an MPC baseline that uses privileged information, including object mass, size, and friction (identified offline), and object pose (estimated via AprilTags). At each timestep, MPC solves (1) in a receding-horizon manner, running at the same frequency as the teacher policy.

Throughout our experiments, we aim to address the following questions:

1. 1.

   Do demonstrations generated by CITO facilitate more effective and efficient learning?

2. 2.

   How does the teacher policy’s performance vary with different demonstrations?

3. 3.

   How robust is the teacher policy compared to a baseline model-based method?

4. 4.

   How accurately can the student estimator predict the privileged information?

5. 5.

   Can the trained policies be successfully transferred to real-world hardware experiments?

Do demonstrations generated by CITO facilitate learning? Across the two tasks, we compare RL performance using different types of demonstrations, corresponding to the different reward formulations in (2). RL with kinematic demonstrations is comparable to prior works such as [40, 41], which only consider kinematically feasible trajectories. Overall, RL with dynamics-based demonstrations achieves the fastest learning as shown in Fig.?2. In particular, in the pivoting without external wall task, neither vanilla RL nor kinematics-conditioned RL was able to learn the skill. We attribute this to the task’s tighter feasible action space. In contrast, dynamics-conditioned RL successfully learns the skill, benefiting from enriched demonstration with contact information.

How does the teacher policy’s performance vary with different demonstrations? For the pivoting-with-wall task, we deploy both kinematics- and dynamics-conditioned RL policies on a real system using a box of mass $110\text{\,}\mathrm{g}$. During deployment, we vary the mass values used as privileged information. Table?1 shows the success rates over three trials. We observe that dynamics-conditioned RL consistently outperforms kinematics-conditioned RL. While both policies are trained with access to privileged information, the dynamics-conditioned policy benefits from demonstrations that include contact force references. This enables the policy to learn physically grounded interaction behaviors during training, leading to greater robustness against variations in dynamic properties. In contrast, the kinematics-conditioned policy is trained with demonstrations that satisfy only geometric feasibility, making it more sensitive to changes in object properties. These results highlight the importance of dynamics-aware demonstrations in contact-rich manipulation tasks.

How accurately can the student estimator predict the privileged information? We deploy the trained student estimator and the teacher policy in MuJoCo and collect both the ground-truth privileged information and the corresponding student estimator’s predictions. Representative results are shown in Fig.?3, demonstrating that our student estimator can successfully predict the privileged information with reasonable accuracy.

Hardware Experiments. We deploy our teacher policy and student estimator on the real robot using zero-shot sim-to-real transfer. Overall, the policy successfully completes the desired task without access to privileged information as shown in Fig.?4.

In this paper, we present a framework for learning closed-loop controllers and estimators for contact-rich pivoting manipulation. We first leverage CITO to generate high-quality demonstrations, including object and robot states, contact forces, and extrinsic contact location. Then, we perform demonstration-guided RL using these demonstrations for training a teacher policy, enabling sample-efficient learning. Furthermore, we train a student estimator using only proprioception, vision, and force sensing, in order to predict the privileged information the teacher policy uses. Our framework is evaluated over several tasks, including the comparison against several baselines, and achieves successful zero-shot sim-to-real transfer in real-world experiments.

Our work has the following limitations. First, we evaluate our framework exclusively on the pivoting task and do not demonstrate results for other non-prehensile manipulation tasks such as pushing and sliding. This choice was intentional to isolate and analyze key system components. However, our method does not assume task-specific priors and is applicable to a broader range of non-prehensile tasks, as long as CITO can generate dynamically feasible demonstrations, which is possible via the approach in [57] or other CITO methods such as [5]. Extending the framework to a multi-task setup or evaluating generalizing across different manipulation tasks remains a promising direction for future work.

Second, all evaluations in this work are performed on convex objects (e.g., boxes), and we do not report results for non-convex geometries. While none of our framework’s modules rely on convexity assumptions, handling non-convex objects introduces additional complexity in contact reasoning. A promising future direction is to train both the teacher policy and student estimator over a distribution of object shapes, enabling generalization across object geometries.

Third, we assume that objects are rigid and non-articulated in this work. This limitation arises from the nature of CITO, which our method relies on for generating demonstrations. The CITO we used in this paper [57] or other CITO methods [74, 19] do not support such dynamics. As a result, the performance may decrease when handling deformable or articulated objects. Extending CITO to support such dynamics, potentially through differentiable simulation, would be a valuable extension.

Fourth, we restrict our focus to quasistatic manipulation in SE(2), which limits the applicability of our proposed framework. Other CITO methods (e.g., [19, 5]) also rely on quasistatic manipulation model assumption. Throughout this work, we rely on this assumption. In particular:

• ?

  The model-based planner we used in this paper [57] operates in SE(2) manipulation due to its inability to model 3D contact dynamics.

• ?

  We leverage the SE(2) assumption to facilitate sim2real transfer, using segmentation to simplify object pose estimation.

To overcome the first limitation, the planner could be extended to support 3D contact dynamics. For the second, incorporating additional camera views to obtain segmentation masks from multiple angles would help lift the restriction to planar manipulation.

Fifth, we empirically observe that policy learning becomes significantly more challenging when the range of domain randomization over table and wall coefficients increases. This is expected, as higher friction can lead to sticking contact while lower friction leads to sliding contact, resulting in multi-modal interaction behavior. In such cases, a richer policy representation, beyond a basic MLP, may be necessary to achieve efficient learning.

Finally, during real-world deployment, we occasionally observed slight object slip (i.e., incipient slip [75, 76, 22]) relative to the robot, resulting in task failure. This issue is quite challenging: the slip must be large enough to produce detectable changes in sensor signals (e.g., vision or force), allowing the student estimator to recognize it, yet small enough to avoid complete contact loss. This limitation is not a significant issue for other works focused on table-top manipulation [5], since objects are inherently stable. Addressing this limitation would likely require higher-resolution sensing or slip-specific estimation modules—for example, integrating visuotactile sensing (e.g., GelSight [77]) or augmenting the student model with incipient slip prediction capabilities.