Position SummaryAs the Robotics System Integration Engineer at Foundation Model Team, you will deploy large-scale multimodal models that integrate VLA (vision, language, and action) components for real-world robotic manipulator applications. To train the VLA models, you will develop an automated data collection pipeline via teleoperation and augment the collected datasets using simulators—beyond lab-scale or proof-of-concept demos.
Key Responsibilities
- Real-World Robot Deployment
- Real-Time Control Integration Work closely with our robotics control team to ensure model outputs align with real-time actuation requirements, bridging deep learning inference and embedded controllers.
- Performance Evaluation & Safety Checks Establish rigorous evaluation protocols (including safety, accuracy, and autonomy metrics) to validate VLA models in industrial or field environments.
- Optimization Package and deploy trained policies onto embedded compute platforms (NVIDIA Jetson or similar), ensuring low-latency inference and reliable control signals.
- Continuous Field Optimization Work hand-in-hand with hardware teams and site operators to diagnose issues and optimize inference for new or unexpected scenarios.
- Automated Data Collection & Data Augmentation
- Automated Data Collection Collaborate with the teleoperation software team to design automated data collection strategies, ensuring high-quality vision and operator-action sequences are captured and automatically saved to the cloud (potentially for use in MLOps workflows).
- Multimodal Annotation Implement processes for labeling or inferring language-based instructions, sensor metadata, and contextual cues from unstructured teleoperation logs.
- Data Augmentation Augment automatically collected datasets via teleoperation in real industrial or field environments by leveraging photorealistic physics simulators.
Qualifications
- Technical Skills
- 3+ years of experience in modern C++ (C++14 or above) and python
- Proficiency using ROS1/2 and embedded edge hardware (e.g., Jetson).
- Strong background in robot control (e.g., bilateral/leader-follower control), with demonstrated ability to handle complex, multi-degree-of-freedom robots when inferring from foundation models
- Experience handling massive datasets, including vision, language, and force sensor data, and uploading them from edge robots to the cloud via networked systems
- Experience with photorealistic physics simulators for robots (e.g., Isaac Sim) for data augmentation and fine-tuning with reinforcement learning
- Demonstrated success in deploying real robot systems (not just simulation).
- Demonstrated ability to integrate hardware, software, and AI/ML components, addressing end-to-end system considerations (safety, reliability, performance).
- Proven ability to troubleshoot and optimize hardware-software integration under real-world conditions.
- Ownership mentality: Takes responsibility for outcomes and problem-solves proactively.
- User-Centric Mindset: Demonstrates the ability to understand how diverse stakeholders (including end users, partners, and internal teams) will interact with the product, envision optimal workflows, communicate these concepts clearly to both technical and non-technical audiences, and translate them into actionable technical requirements.
- Comfortable in a performance-driven environment (high rewards for results, potential demotion for underperformance).
- Communication skills in English; Japanese proficiency is a plus.
Professional Experience
Soft Skills & Culture Fit
The role assignment may be modified based on organizational requirements. This could include changes to your job responsibilities, reporting structure, or team assignment.
求人を報告
