Software Engineer

in Information Technology
  • New York City, New York View on Map
  • Salary: $65.00 - $65.00
Contract

Job Detail

  • Experience Level Mid Level
  • Degree Type Other
  • Employment Contract
  • Working Type Remote
  • Job Reference 0000022829
  • Salary Type Hourly
  • Industry Software
  • Selling Points

    Contribute to cutting-edge AI coding evaluations in a fully remote role. Collaborate with innovative technologies to enhance coding agent capabilities. Flexible work arrangements and impactful engineering opportunities.

Job Description

Software Engineer Overview

  • As a Software Engineer, contribute to evaluating AI coding agents within realistic codebases.
  • Design and test tasks that assess AI agent performance against professional standards.
  • Engage in long-form, self-directed work across multiple programming languages and problem types.
  • Work remotely without production on-call or shift work requirements.
  • Leverage your engineering judgment to grade AI agent outputs effectively.
  • Collaborate with cutting-edge AI technologies to enhance coding agent capabilities.
  • Contribute to creating rubrics and tests defining correct deliverables for AI agents.
  • Analyze and improve AI coding agent outputs by identifying and flagging failures.

Software Engineer Key Responsibilities & Duties

  • Evaluate AI coding agents' performance on tasks derived from real-world engineering practices.
  • Design and implement tasks such as bug fixes, feature development, and refactoring.
  • Conduct pull request reviews and debugging write-ups based on logs and traces.
  • Compare outputs from AI agents, determining quality and areas of improvement.
  • Develop tests and rubrics to evaluate deliverables against professional standards.
  • Identify and document issues such as brittle changes, missed edge cases, and fabricated files.
  • Collaborate with AI/LLM tools to enhance coding agent performance.
  • Work independently to deliver high-quality assessments and improvements.

Software Engineer Job Requirements

  • Professional experience in software engineering, with 2+ years preferred.
  • Proficiency in at least one common programming stack such as Python, JavaScript, or Java.
  • Comfortable with terminal operations, debugging, and unfamiliar codebases.
  • Experience with Docker and CI tools is advantageous.
  • Strong written communication skills for explaining technical issues clearly.
  • Familiarity with AI/LLM tools, with user-level expertise being sufficient.
  • Ability to work independently and remotely, managing self-directed tasks effectively.
  • Depth in specialized areas such as frontend, backend, or mobile engineering is a plus.
  • ShareAustin:

Related Jobs

  • Advance your career in a hybrid IT role with cutting-edge technologies. Gain hands-on experience in cloud computing and AI-enabled tools. Collaborate on modernization projects and enhance technical expertise.