Turn it in here

Be michelle - 3.5.

Watch the video associated with unit 3:5 on page 118. Record yourself as you take on the role of Michelle (B). Really answer the questions, do not just copy what Michelle is saying. You may find it helpful to have the video running as you record yourself to help with timing and make it feel like a real conversation.

asl homework 3.5

Pardon Our Interruption

As you were browsing something about your browser made us think you were a bot. There are a few reasons this might happen:

  • You've disabled JavaScript in your web browser.
  • You're a power user moving through this website with super-human speed.
  • You've disabled cookies in your web browser.
  • A third-party browser plugin, such as Ghostery or NoScript, is preventing JavaScript from running. Additional information is available in this support article .

To regain access, please make sure that cookies and JavaScript are enabled before reloading the page.

Pardon Our Interruption

As you were browsing something about your browser made us think you were a bot. There are a few reasons this might happen:

  • You've disabled JavaScript in your web browser.
  • You're a power user moving through this website with super-human speed.
  • You've disabled cookies in your web browser.
  • A third-party browser plugin, such as Ghostery or NoScript, is preventing JavaScript from running. Additional information is available in this support article .

To regain access, please make sure that cookies and JavaScript are enabled before reloading the page.

HCCS Learning Web

  • Houston Community College
  • Eagle Online

HCCS Learning Web

  • Lyman Mann III

American Sign Language II (SGNL 1402 34882)

Course Syllabus

American Sign Language (ASL) Beginning II

 

 Fall 2014, CRN 34882

 Lyman (Joseph) Mann

 

  or

Central Campus, San Jacinto Building, room 192.6

By appointment only

 SGNL 1402-01 Central Campus, SE-San Jacinto BDG,  Rm. 188,  M/W 12:30 pm-3:00 pm

Credit Hours

4.00

 

Lecture Hours

3.00

 

Laboratory Hours

2.00

 

Total Course Contact Hours

Continuing Education Units (CEU): if applicable

Course Length (number of weeks)

Type of Instruction

Lecture in American Sign Language, videotapes, quizzes, videotaping of signing skills, receptive/expressive final exams. No voice is used in ASL classes therefore, we expect that students will adhere to this regulation also.

Course Description:

ASL II develops receptive and expressive ability and allows for recognition and demonstration of more sophisticated grammatical features of American Sign Language (ASL). This course increases fluency and accuracy in fingerspelling and numbers, and provides opportunities for interaction within the deaf community.

Student must pass the final exam Benchmark with a “B” or better. If you do not pass the final with a “B” or better, you must take mandatory tutoring and re-test the final exam.

Course Continuing Education Units (CEU): If applicable

Course Prerequisite(s)

SGNL 1301 or

Course Description: ACGM or WECM

Develops receptive and expressive ability and allows recognition and demonstration of more sophisticated grammatical features of American Sign Language (ASL).  Increases fluency and accuracy in fingerspelling and numbers.  Encourages opportunities for interaction within the Deaf community.  Increases fluency and accuracy in spatial organization and techniques, ASL narrative skills/conversation, facial expression, and description skills using classifiers.

Course Description: HCC Catalog Description

Academic Discipline/CTE Program Learning Outcomes

  • Develop receptive and expressive skills in American Sign Language and Fingerspelling;
  • Develop knowledge and awareness of the differences between the Deaf culture/deaf community and the hearing community;
  • Accurately interpret and transliterate between ASL and English in a variety of settings:  face-to-face, small group settings, monologue and/or large group settings;
  • Apply professional standards, practices, and ethics, not limited to the tenets of the Code of Professional Conduct, to their work

Course Student Learning Outcomes (SLO): 4 to 7

Exhibit continued development of skills in expressive and receptive ASL communications, including fingerspelling and numbers

Identify and demonstrate more complex grammatical features of ASL, including non-manual markers, inflected signs, spatial referencing, classifiers and temporal sequencing

Initiate, conduct and terminate short/medium length context specific conversations in ASL

Demonstrate self-generated short stories and narratives

Demonstrate an expanded core vocabulary

Learning Objectives (Numbering system should be linked to SLO - e.g., 1.1, 1.2, 1.3, etc.)

Learning Outcomes 1: Using American Sign Language, the student will be able to demonstrate and comprehend information presented in American Sign Language, based on Signing Naturally, Unit 7,”Giving Directions” curriculum with a minimum of 70% mastery .

Performance Objectives for this Outcome: Upon completion of Unit 7 , the student will be able to use ASL receptively and expressively and receptively to:

1.1    specify directions within a multi-story building and surrounding outdoor locations using a common reference point 1.2 explain and comprehend location directions going from general to specific 1.3 demonstrate and comprehend ordinal numbers from “first” on up using appropriate form, hand shape, palm orientation and movement 1.4 explain a problem and ask for alternate locations 1.5 ask/tell basic needs and wants related to hunger and thirst using vending machine snack vocabulary

1.6 identifies a fingerspelled word within a signed sentence.

Methods of Measurement : Written assignments, quizzes, class participation, homework, expressive video assignment and final exam.

Learning Outcomes 2 : Using American Sign Language, the student will be able to demonstrate and comprehend information presented in Unit 8, “Describing Others” , based on the Signing Naturally, Unit 8 curriculum with a minimum of 70% accuracy .

Performance objectives for this outcome: Upon completion of Unit 8 , the student will be able to use ASL receptively and expressively to:

2.1 demonstrate and comprehend the interactive process used to identify people who are present or not present

2.2 ask/tell “who” in real world orientation – identify others by race, physical appearance, clothing and clothing patterns and accessories

2.3 correctly produce and comprehend multiples of 10 and 11 using appropriate form, hand shape, palm orientation and movement

Methods of Measurement: Written assignments, quizzes, class participation, homework, expressive video assignment and final exam

Learning Outcomes 3: Using American Sign Language, the student will be able to demonstrate and comprehend information presented in Unit 9 , “Making Requests”, based on Signing Naturally, Unit 9 curriculum.

Performance objectives for this outcome: Upon completion of Unit 9 , the student will be able to use ASL receptively and expressively to:

3.1 ask/answer requests for assistance including stating the problem and accepting and/or declining the solution

3.2 formulate and comprehend numbers related to money using appropriate form, hand shape, palm orientation and movement from one cent up to one dollar

3.3 demonstrate and comprehend inflecting and spatial verbs

3.4 Identify and demonstrate vocabulary related to objects in a variety of locations

3.5 specify and comprehend stages of completion

3.6 successfully re-tell a story in ASL by shadowing/mirroring the storyteller

Methods of Measurement: Written assignments, written quizzes, class participation, homework and final exam

Learning Outcomes 4: Using American Sign Language, the student will be able to demonstrate and comprehend information presented in Unit 10 , “Talking and family and Occupations”, based on Signing Naturally, Unit 10 curriculum with a minimum of 70% mastery .

Performance objectives for this outcome: Upon completion of Unit 10, the student will be able to use ASL receptively and expressively to:

4.1 demonstrate and comprehend personal and possessive pronouns

4.2 correctly produce and comprehend numbers representing ages of people using appropriate form, palm orientation, hand shape and movement

4.3 correctly produce and comprehend ranking of ages in a person’s family

4.4 demonstrate and comprehend vocabulary related to workplaces, hospitals and occupations

4.5 identify and demonstrate persons of authority in an occupation

4.6 explain and comprehend if people in a relationship “get along” or not

Methods of Measurement: Written assignments, written quizzes, class participation, homework, expressive video about self and final exam

Learning Outcomes 5: Using American Sign Language, the student will be able to demonstrate and comprehend information presented in Unit 11, “Attributing Qualities to Others” , based on Signing Naturally, Unit 11 curriculum with a minimum of 70% accuracy.

Performance objectives for this outcome: Upon completion of Unit 11 the student will be able to use ASL receptively and expressively to:

5.1 demonstrate and comprehend contradicting opinions using the sign “but”; include physical descriptions, positive and negative attributes and mannerisms

5.2 demonstrate and comprehend one character role shifting correctly using eye gaze and head position shift

5.3 produce and comprehend numbers 67 through 98 using appropriate form, hand shape, palm orientation and movement

5.4 demonstrate and comprehend contrastive structure with emphasis on adjective antonyms

5.5 demonstrate and comprehend basic animal (pets) vocabulary

Learning Outcomes 6: Using American Sign Language, the student will be able to demonstrate and comprehend information presented in Unit 12, “Talking about Routines” , based on Signing Naturally, Unit 12 curriculum with a minimum of 70% accuracy.

Performance objectives for this outcome: Upon completion of Unit 12 , the student will be able to use American Sign Language receptively and expressively to:

6.1 explain and comprehend daily routines including resolving conflicts in scheduling

6.2 correctly produce and comprehend time including clock numbers and combing hour and minute information using appropriate form, hand shape, palm orientation and movement

6.3 demonstrate and comprehend the signs for frequency in daily routines. This includes “every” sign for weekly or monthly activities and length of time of an activity, “approximate” sign, and “before” and “after” signs

SCANS and/or Core Curriculum Competencies: If applicable

Identify and demonstrate more complex grammatical features of ASL, including non-manual markers, inflected signs, spatial referencing, classifiers, and temporal sequencing

Initiate, conduct, and terminate short/medium length context specific conversations in ASL

SCANS Workplace Competencies are defined in five areas: (a) resources, (b) interpersonal skills, (C) information, (d) systems, and (e) technology.

The following SCANS competencies will be included in this course:

Resources: A worker must identify, organize, plan, and allocate resources effectively. C1- Time: select goal-relevant activities, rank them, allocate time, and prepare and follow schedules.

Information: A worker must be able to acquire and use information. C5- Acquire and evaluate Information. C6- Organize and maintain information. C7- Interpret and communicate Information.

Interpersonal Skills: A worker must work with others effectively. C14- Work with Diversity: work well with men and women from diverse backgrounds.

Systems: A worker must understand complex interrelationships. C16- Monitor and correct performance: distinguish trends, predict impacts on system operations, diagnose deviations in systems' performance and correct malfunctions.

Basic Skills: A worker must read, write, perform arithmetic and mathematical operations, listen, and speak effectively.

The following foundation skills will be included in this course:

Foundation Skills are defined in three areas:

(a) Basic skills, (b) thinking skills, and (C) personal qualities. F1- Reading: locate, understand, and interpret written information in prose and in documents such as manuals, graphs, and schedules. F2- Writing: communicate thoughts, ideas, information, and messages in writing, and create documents such as letters, directions, manuals, reports, graphs, and flow charts. F4- Listening: receive, attend to, interpret, and respond to verbal messages and other cues. F5- Speaking: organize ideas and communicate orally. F6- Speaking-organizes ideas and communicate orally.

Thinking Skills: A worker must think creatively, make decisions, solve problems, visualize, know how to learn, and reason effectively. These skills include: F10- Knowing how to learn: use efficient learning techniques to acquire and apply new knowledge and skills.

Personal Qualities: A worker must display responsibility, self-esteem, sociability, self-management, integrity, and honesty. F11- Knowing How to Learn-uses efficient learning techniques to acquire and apply new knowledge and skills. F12- Responsibility: exert a high level of effort and persevere toward goal attainment. F13- Self-esteem: believe in one's own self-worth and maintain a positive view of oneself. F14- Sociability: demonstrate understanding, friendliness, adaptability, empathy, and politeness in group settings. F15- Self-management: assess oneself accurately, set personal goals, monitor progress, and exhibit self-control. F16- Integrity and honesty: choose ethical courses of action. F17- Integrity/Honesty-Chooses ethical course of action.

Course Calendar

Week

Number

Activities and Assignment

Objectives and Details

1 (M/W)

8/25/14

Introduction, syllabus, ACE, Email, Classroom Management, Online Eagle, & Class Requirement

8/27/14

Unit 7 “What’s Around Here?” And “Giving Directions on Different Floors”, pages 124-125

2

Sept

9/3/14

Unit 7 "Giving Directions on the Same Floor” and “Where is it”, page 125-126

 

9/8/14

Unit 7 “Where is Houston?”

Review page 125, Giving Directions on the Same Floor.  The goal is to emphasize how to establish a reference point, and to review and practice giving specific directions and locations.

“Where on Campus?” , page 127          

3

9/10/14

Unit 7 “Excuse me, Where’d You Get that Candy?” page 131.  “Expressing Uncertainty”, pages 132–133.  

“Snack Time”, page 128 

“I’m Thirsty”, Where Can I get a Drink?”, page 130

Unit 7 Vocabulary Review

Unit 7 Self-Assessment based on benchmark    

 

9/15/14

Unit 8 “Describing Style and Pattern of Clothing”, page 162

Review basic colors, and clothing items but also add quality of a color, LIGHT, DARK, LOUD, BRIGHT

Breakaway, page 161 (Numbers in multiples of 5, 10, and 11. Play the game 99 (see ASL I, 527)          

4

9/17/14

Unit 8 “Interaction-Relay a message” (Follow up Activity from homework)

Describing Animals expand by introducing Body Part Classifiers, Semantic Classifiers, and DCLs to describe animals.          

 

9/22/14

Unit 8 Review classifiers and play games using animals and people.

5

9/24/14

Unit 8 Continue reviewing classifiers

 

9/29/14

Units 7 and 8 Review

Unit 8 Vocabulary Review

Unit 8 Self-Assessment based on benchmark

6

10/1/14

 

10/6/14

Group Discussion/Feedback on Exam

Unit 9 “Give and Take”, page 109, Student Workbook.

7

10/8/14

Unit 9 “Making Requests, page 109, Student Workbook.

10/13/14

Unit 9 Group Discussion on ABC Gum Story.  Students break into 4 groups and practice re-telling story.

“What’s Out There”, page 199

8

10/15/14

Review Unit 9

Unit 9 Vocabulary

Unit 9 Self-Assessment based on benchmark

Classifiers Review

 

10/20/14

Unit 10 Review ASL Grammar Rules

Midterm - Classifiers

9

10/22/14

Unit 10 Review and expand  ASL Grammar Rules

“Ask/Tell How Old”, page 219-221

“Ranking by Age”, page 221

 

10/27/14

Unit 10 “Establishing Relationship”, page 224

“US-TWO, THEY-TWO,” pages 225-226

“Which Occupation Is It?”, page 227

10

10/29/14

Units, 9 & 10 Review

Unit 10 Vocabulary

Unit 10 Self-Assessment based on benchmark

 

11/3/14

11

11/5/14

Group Discussion/Feedback on Exam

Unit 11 “Describing Personal Qualities”, page 254-257

Dialogue Practice, page 260

11/10/14

Unit 11 Guessing Game: Personality, page 261

Desirable Personal Qualities for Specific Jobs, page 261

12

11/12/14

Unit 11 “A Fishy Story”, Student Workbook, page 141

 

11/17/14

Unit 12 “Weekly and Monthly Routines”, page 279 (Review ASL I, Unit 5)

“Solving Conflicts”, page 282-283 then do Student Workbook,  Language in Action, page, 150

13

11/19/14

Unit 12 “Fill in the Blanks: Daily Activities”, page 282

“What’s Your Alibi?”, page 283

 

11/24/14

Safe Keeping (Student Workbook, page 154)

Unit 12 Vocabulary

Unit 12 Self-Assessment based on benchmark

14

11/26/14

12/1/14

Group Discussion/Feedback on Exam

5 Parameters

15

12/3/14

Cumulative Review

12/8/14

Cumulative Review

16

Dec 10, 2014

Instructional Methods

 Lecture in American Sign Language (ASL), videotapes, quizzes, videotaping of signing skills, written midterm and final exams. No voice is used in ASL classes, therefore, we expect for students to adhere to this regulation also.

Functional – Notional Approach

The authors of Signing Naturally chose an approach which focuses on the “functions” of communicative purposes of people’s every interaction. They emphasize functions that help students establish and maintain social relationships. “Activities are varied and allow students to use different learning strategies to practice what they have learned. The indirect benefit of these situations for the student is the development of cultural awareness and cross-cultural adjustment skills.”

Teacher’s edition, Signing Naturally Level 1 1998

Student Assignments

Practice Conversation 1 from Student Workbook from Language in Action, page 77.

Practice Conversation 2 from Student Workbook from Language in Action, page 77.

Students read Grammar Notes, pages 78-79 and Grammar Practice, page 79.

Pair Practice, Practice 1 and 2, page 80

The Candy Bar, page 80

Practice Conversation 1 from Student Workbook from Language in Action, page 89.

Practice Conversation 2 from Student Workbook from Language in Action, page 89.

Students read Grammar Notes, pages 90-91 and Grammar Practice, page 91-92.

Pair Practice, Practice 1 and 2, page 92

Personal Data, page 93

Interaction-Missing Person

Practice Conversation 1 from Student Workbook from Language in Action, page 106.

Practice Conversation 2 from Student Workbook from Language in Action, page 106.

Students read Grammar Notes, pages 90-91 and Grammar Practice, page 107-109.

What are three different verb types?, page 108

Review Gum Story from ASL I, Unit 6 then develop ABC gum story

Review Classifiers

Review ASL Grammars.

Practice Conversation 1 from Student Workbook from Language in Action, page 124.

Practice Conversation 2 from Student Workbook from Language in Action, page 124.

Students read Grammar Notes, pages 90-91 and Grammar Practice, page 125-126.

Comprehension, page 126

What is different sign between personal and possessive pronouns?, page 125

Practice Conversation 1 from Student Workbook from Language in Action, page 138.

Practice Conversation 2 from Student Workbook from Language in Action, page 138.

Students read Grammar Notes, page 139 and Grammar Practice, page 140.

Comprehension, page 141

Grammar Practice from Student Workbook, page 151

Calendar, Part 2 from Student Workbook, page 152

Comprehension, “What Time?” from Student Workbook, page 153

Student Assessment(s)

Pre/post testing

Participation (including laboratory)

Self-assessment (including peer reviews)

Games and interactive exercises

Objective assessment (expressive and receptive skills on videotape)

Instructor's Requirements

  Student Involvement

The faculty members of the Department of American Sign Language and Interpreter Training are committed to your successful completion of our classes without lowering the college’s academic standards. I understand that students face additional pressures from work and family, as well as have other obligations outside of their academic pursuits. I realize that at times issues beyond the control of a student interfere with class requirements.

If you experience any circumstance that has a negative impact on your participation in this course, please make me aware of it as soon as possible. I may be able to assist or accommodate your particular circumstance. Do not wait until the end of the semester to ask for advice. Communication between students and instructors can be quite valuable. 

Program/Discipline Requirements: If applicable

Global Awareness

This class will encourage an understanding of the importance of diversity and difference in the college, the community, and the country.

HCC Grading Scale

A = 900-1000

 

4 points per semester hour

B = 800-899

 

3 points per semester hour

C = 700-799

 

2 points per semester hour

F = 699 and below

 

1 point per semester hour

W(Withdrawn)

 

0 points per semester hour

AUD (Audit)

 

0 points per semester hour

To compute grade point average (GPA), divide the total grade points by the total number of semester hours attempted. The grade "I" does not affect GPA.

Instructor Grading Criteria

Grade will be determined by the following

Details

Points

Points you earned

5 Small Projects

(5 Projects, 10 points each total)

1# = Your Home Plan (Units 7&9)

2# = Potato Head (Unit 8)

3# = Classifiers

4# = Re-tell Story “Forever Alone! By Letsgofly08”

5# = ASL Grammars

50

 

 

 

 

Homework

 

15

 

ELGS3 (survey)

 

10

 

Test 1 (Units 7 & 8)

 

75

 

Test 2 (Unit 9 & 10)

 

75

 

Test 3 (Units 11 & 12)

 

75

 

Final Exam

 

700

 

 

Total Points

1000

 

Grades         900-1000     A

                     800-899       B

                     700-799       C

                      600-699      D

                     Below 599    F

The lab assignments will be watching a video and writing a short summary of what it said and also answering questions which will be turned in.

Instructional Materials

Signing Naturally, Level I Student Workbook and videotext,  Units 7-12 (workbook with DVD)

Flip video camera

YouTube Account

One mini DVD-R

HCC Policy Statement: ADA and Academic Honesty

Access Student Services Policies on their Web site: http://hccs.edu/student-rights

HCC Policy Statement: Student attendance, 3-peaters, withdrawal deadline

  Attendance:   Students are expected to attend all classes and labs regularly. Students are responsible for [any and all] materials covered during their absences, and it is the student’s responsibility to consult with the professors for make-up assignments. A student may be dropped from a course for excessive absences in excess of 12.5% of the hours of instruction. For example: For a three-credit hour lecture, a student may be dropped after six hours of absence.  HCCS professors cannot assign a “W” for any student after the official withdrawal date. “Administrative withdrawals are the discretion of the professor. If you are doing poorly in the class, but you have not contacted your professor to ask for help, and you have not withdrawn by the official withdrawal date, it will result in you receiving a grade of “F” in the course.

Course Withdrawals-First Time Freshmen Students-Fall 2007 and Later: Under Section 51.907 of the Texas Education Code “an institution of higher education may not permit a student to drop more than six courses, including any course a transfer student has dropped at another institution of higher education.” Beginning in fall 2007, the Texas Legislature passed a law limiting first time entering freshmen to no more than SIX total course withdrawals throughout their educational career in obtaining a certificate and/or degree. 

Course Withdrawals: Be sure you understand HCC policies about dropping a course. It is the student’s responsibility to withdraw officially from a course and prevent an “F” from appearing on the transcript. If you feel that you cannot complete this course, you will need to withdraw from the course prior to the final date of withdrawal.   Before, you withdraw from your course; please take the time to meet with the instructor to discuss why you feel it is necessary to do so. The instructor may be able to provide you with suggestions that would enable you to complete the course.  Your success is very important. 

If you plan on withdrawing from your class, you MUST contact a HCC counselor or your professor prior to withdrawing (dropping) the class for approval and this must be done PRIOR to the withdrawal deadline to receive a “W” on your transcript.  **Final withdrawal deadlines vary each semester and/or depending on class length, please visit the online registration calendars, HCC schedule of classes and catalog, any HCC Registration Office, or any HCC counselor to determine class withdrawal deadlines.  Remember to allow a 24-hour response time when communicating via email and/or telephone with a professor and/or counselor.  Do not submit a request to discuss withdrawal options less than a day before the deadline.  If you do not withdraw before the deadline, you will receive the grade that you are making in the class as your final grade. 

Early Alert Program:   To help students avoid having to drop/withdraw from any class, HCC has instituted an Early Alert process by which your professor may “alert” you and HCC counselors that you might fail a class because of excessive absences and/or poor academic performance.  It is your responsibility to visit with your professor or a counselor to learn about what, if any, HCC interventions might be available to assist you – online tutoring, child care, financial aid, job placement, etc. – to stay in class and improve your academic performance. 

Repeat Course Fee :  The State of Texas encourages students to complete college without having to repeat failed classes.  To increase student success, students who repeat the same course more than twice, are required to pay extra tuition. 

Distance Education and/or Continuing Education Policies

Access DE Policies on their Web site: http://de.hccs.edu/Distance_Ed/DE_Home/faculty_resources/PDFs/DE_Syllabus.pdf Access CE Policies on their Web site: http://hccs.edu/CE-student-guidelines

EGLS 3 -Evaluation for Greater Learning Student Survey System

At Houston Community College, professors believe that thoughtful student feedback is necessary to improve teaching and learning. During a designated time, you will be asked to answer a short online survey for research-based questions related to instruction. The anonymous results of the survey will be made available to your professors and division chairs for continual improvement of instruction. Looks for the  EGLS3 as part of the Houston Community College Student System online near the end of the term.

Course Information

Try AI-powered search

  • GPT, Claude, Llama? How to tell which AI model is best

Beware model-makers marking their own homework 

An illustration showing a lineup of robots of different shapes and sizes.

Your browser does not support the <audio> element.

W hen Meta , the parent company of Facebook, announced its latest open-source large language model ( LLM ) on July 23rd, it claimed that the most powerful version of Llama 3.1 had “state-of-the-art capabilities that rival the best closed-source models” such as GPT -4o and Claude 3.5 Sonnet. Meta’s announcement included a table, showing the scores achieved by these and other models on a series of popular benchmarks with names such as MMLU, GSM 8 K and GPQA .

On MMLU , for example, the most powerful version of Llama 3.1 scored 88.6%, against 88.7% for GPT -4o and 88.3% for Claude 3.5 Sonnet, rival models made by Open AI and Anthropic, two AI startups, respectively. Claude 3.5 Sonnet had itself been unveiled on June 20th, again with a table of impressive benchmark scores. And on July 24th, the day after Llama 3.1’s debut, Mistral, a French AI startup, announced Mistral Large 2, its latest LLM , with—you’ve guessed it—yet another table of benchmarks. Where do such numbers come from, and can they be trusted?

Having accurate, reliable benchmarks for AI models matters, and not just for the bragging rights of the firms making them. Benchmarks “define and drive progress”, telling model-makers where they stand and incentivising them to improve, says Percy Liang of the Institute for Human-Centred Artificial Intelligence at Stanford University. Benchmarks chart the field’s overall progress and show how AI systems compare with humans at specific tasks. They can also help users decide which model to use for a particular job and identify promising new entrants in the space, says Clémentine Fourrier, a specialist in evaluating LLM s at Hugging Face, a startup that provides tools for AI developers.

But, says Dr Fourrier, benchmark scores “should be taken with a pinch of salt”. Model-makers are, in effect, marking their own homework—and then using the results to hype their products and talk up their company valuations. Yet all too often, she says, their grandiose claims fail to match real-world performance, because existing benchmarks, and the ways they are applied, are flawed in various ways.

One problem with benchmarks such as MMLU (massive multi-task language understanding) is that they are simply too easy for today’s models. MMLU was created in 2020 and consists of 15,908 multiple-choice questions, each with four possible answers, across 57 topics including maths, American history, science and law. At the time, most language models scored little better than 25% on MMLU , which is what you would get by picking answers at random; Open AI ’s GPT -3 did best, with a score of 43.9%. But since then, models have improved, with the best now scoring between 88% and 90%.

asl homework 3.5

This means it is difficult to draw meaningful distinctions from their scores, a problem known as “saturation” (see chart). “It’s like grading high-school students on middle-school tests,” says Dr Fourrier. More difficult benchmarks have been devised— MMLU -Pro has tougher questions and ten possible answers rather than four. GPQA is like MMLU at P h D level, on selected science topics; today’s best models tend to score between 50% and 60% on it. Another benchmark, M u SR (multi-step soft reasoning), tests reasoning ability using, for example, murder-mystery scenarios. When a person reads such a story and works out who the killer is, they are combining an understanding of motivation with language comprehension and logical deduction. AI models are not so good at this kind of “soft reasoning” over multiple steps. So far, few models score better than random on M u SR.

MMLU also highlights two other problems. One is that the answers in such tests are sometimes wrong. A study carried out by Aryo Gema of the University of Edinburgh and colleagues, published in June, found that, of the questions they sampled, 57% of MMLU ’s virology questions and 26% of its logical-fallacy ones contained errors. Some had no correct answer; others had more than one. (The researchers cleaned up the MMLU questions to create a new benchmark, MMLU -Redux.)

Then there is a deeper issue, known as “contamination”. LLM s are trained using data from the internet, which may include the exact questions and answers for MMLU and other benchmarks. Intentionally or not, the models may be cheating, in short, because they have seen the tests in advance. Indeed, some model-makers may deliberately train a model with benchmark data to boost its score. But the score then fails to reflect the model’s true ability. One way to get around this problem is to create “private” benchmarks for which the questions are kept secret, or released only in a tightly controlled manner, to ensure that they are not used for training ( GPQA does this). But then only those with access can independently verify a model’s scores.

To complicate matters further, it turns out that small changes in the way questions are posed to models can significantly affect their scores. In a multiple-choice test, asking an AI model to state the answer directly, or to reply with the letter or number corresponding to the correct answer, can produce different results. That affects reproducibility and comparability.

Automated testing systems are now used to test models against benchmarks in a standardised manner. Dr Liang’s team at Stanford has built one such system, called HELM (holistic evaluation of language models), which generates leaderboards showing how a range of models perform on various benchmarks. Dr Fourrier’s team at Hugging Face uses another such system, Eleuther AI Harness, to generate leaderboards for open-source models. These leaderboards are more trustworthy than the tables of results provided by model-makers, because the benchmark scores have been generated in a consistent way.

The greatest trick AI ever pulled

As models gain new skills, new benchmarks are being developed to assess them. GAIA , for example, tests AI models on real-world problem-solving. (Some of the answers are kept secret to avoid contamination.) NoCha (novel challenge), announced in June, is a “long context” benchmark consisting of 1,001 questions about 67 recently published English-language novels. The answers depend on having read and understood the whole book, which is supplied to the model as part of the test. Recent novels were chosen because they are unlikely to have been used as training data. Other benchmarks assess models’ ability to solve biology problems or their tendency to hallucinate.

But new benchmarks can be expensive to develop, because they often require human experts to create a detailed set of questions and answers. One answer is to use LLM s themselves to develop new benchmarks. Dr Liang is doing this with a project called AutoBencher, which extracts questions and answers from source documents and identifies the hardest ones.

Anthropic, the startup behind the Claude LLM , has started funding the creation of benchmarks directly, with a particular emphasis on AI safety. “We are super-undersupplied on benchmarks for safety,” says Logan Graham, a researcher at Anthropic. “We are in a dark forest of not knowing what the models are capable of.” On July 1st the company began inviting proposals for new benchmarks, and tools for generating them, which it will co-fund, with a view to making them available to all. This might involve developing ways to assess a model’s ability to develop cyber-attack tools, say, or its willingness to provide advice on making chemical or biological weapons. These benchmarks can then be used to assess the safety of a model before public release.

Historically, says Dr Graham, AI benchmarks have been devised by academics. But as AI is commercialised and deployed in a range of fields, there is a growing need for reliable and specific benchmarks. Startups that specialise in providing AI benchmarks are starting to appear, he notes. “Our goal is to pump-prime the market,” he says, to give researchers, regulators and academics the tools they need to assess the capabilities of AI models, good and bad. The days of AI labs marking their own homework could soon be over. ■

Curious about the world? To enjoy our mind-expanding science coverage, sign up to  Simply Science , our weekly subscriber-only newsletter.

Explore more

Science & technology august 3rd 2024, gene-editing drugs are moving from lab to clinic at lightning speed, how america built an ai tool to predict taliban attacks.

Chinese business goes global

From the August 3rd 2024 edition

Discover stories from this section and more in the list of contents

More from Science & technology

asl homework 3.5

“Raven Sentry” was a successful experiment in open-source intelligence

asl homework 3.5

The promising treatments still face technical and economic hurdles, though

asl homework 3.5

How Ukraine’s new tech foils Russian aerial attacks

It is pioneering acoustic detection, with surprising success

The deep sea is home to “dark oxygen”

Nodules on the seabed, rather than photosynthesis, are the source of the gas

Augmented reality offers a safer driving experience

Complete with holograms on the windscreen

Clues to a possible cure for AIDS

Doctors, scientists and activists meet to discuss how to pummel HIV

COMMENTS

  1. Signing Naturally Unit 3.5 Flashcards

    front door. Study with Quizlet and memorize flashcards containing terms like to reside, to live at, house, city and more.

  2. Bob LoParo's ASL 101 Class

    Be Michelle - 3.5. Watch the video associated with unit 3:5 on page 118. Record yourself as you take on the role of Michelle (B). Really answer the questions, do not just copy what Michelle is saying. You may find it helpful to have the video running as you record yourself to help with timing and make it feel like a real conversation.

  3. 3.1 Tell Where, 3.2 Commands, 3.5 Residences, 3.6 Directions ...

    ASL Homework 3:12 - HOW LONG DOES IT TAKE? Answers. 10 terms. brizzyw5. Preview. Activity 14D ERWC: Furious Questions. 5 terms. Tryin2make_schl_ezer. Preview. Pulitzer Prize Winners - Drama (1961-1985) Teacher 20 terms. varelaspanish. Preview. ss. 6 terms. acioran. Preview. ASL Homework 3:10 - STORY CORNER: "THE ELEVATOR INCIDENT" Answers.

  4. Signing Naturally Unit 3 Answers

    Basic American Sign Language I (ASL 112) 6 Documents. Students shared 6 documents in this course. University College of Staten Island CUNY. Academic year: 2022/2023. Uploaded by: Anonymous Student. ... Homework 3: Minidialogue 1 - ASL class is three hours. Minidialogue 2 - nap is twenty minutes. Minidialogue 3 - Baking cookies takes 10-12 ...

  5. ASL 1

    ASL 1 - 2.4 Homework; ASL 1 - 2.2 Homework; ASL 1 - 1.9 Homework; ASL 4 - Chapter 4 lesson 5 assignments; ASL 1 - Chapter 1 lesson 11 assignments; Preview text ... Course: American Sign Language 1 (CSD 138) 74 Documents. Students shared 74 documents in this course. University: Butler University. Info More info. AI Quiz. AI Tools.

  6. Signing Naturally 3.5 minidialogues.docx

    View Homework Help - HW 3_5 from LING 071 at University of Pennsylvania. 3:5 Minidialog... homework. ASL 3.5 Homework - Google Docs.pdf. Colorado State University, Global Campus. SPH 251. ... View ASL Homework 3.5.docx from ASL 111 at County College of Morris. ASL Homework 3.5 C...

  7. ASL homework 3.5 part 1 (png)

    ASL homework 3.5 part 1.png. School. Wilson College * *We aren't endorsed by this school. Course. AST 101. Subject. English. Date. Apr 30, 2024. Pages. 1. Uploaded by ... so no matter what you're studying, CliffsNotes can ease your homework headaches and help you score high on exams. Quick Links. Literature Notes Study Guides Documents Homework ...

  8. Signing Naturally 3.5.docx

    View Homework Help - Signing Naturally 3.5.docx from ASL 121 at Community College of Denver. Unit 3.5 Mini dialogue 1 1. Is Dale a man or a woman? ... View Homework Help - ASL_HW3.5.pdf from PST 301 at Gallaudet University. Jillian Weeks... homework. asl 3.5.docx. Gallaudet University. ASL OL-3. Primary color. Orange.

  9. ASL 1 Signing Naturally Unit 3:1 and 3:5 Places & Homes

    ASL Homework 3:12 - HOW LONG DOES IT TAKE? Answers. 10 terms. brizzyw5. Preview. ASL Cities and States. 24 terms. wjiaei. Preview. Latin stage 32 vocab. 21 terms. Zachary_Greenberg9. Preview. Medical Terminology Basics. 57 terms. lexyrobinsonccs. Preview. ASL NUMBERS 1-66. Teacher 66 terms. Sara_Sargent24. Preview. consumer rights and ...

  10. Unit 3.5 Worksheet

    View Homework Help - Unit 3.5 Worksheet from ASL 19202 at Kent State University. NAME: Austen Peters 3/4/16 ASL 19201 HOMEWORK 3.5 HOMEWORK 3.5 Mini Dialogue (p 118 to 120) from your textbook Mini

  11. ASL 1 Chapter 3 Bookwork

    HOmework for this unit cole williams asl karlene kischer browne chapter homework minidialogue berkeley who lives there? ben which area near university fremont. ... American Sign Language (Asl) I (CSD 3190) More from: American Sign Language (Asl) I CSD 3190. University of Northern Iowa. 9 Documents.

  12. ASL 3.5 Homework

    ASL 3.5 Homework - Google Docs.pdf - 3.5 ASL Homework... Pages 1. Identified Q&As 1. Total views 32. Colorado State University, Global Campus. SPH. SPH 251. mikeywebbpanda. 5/17/2021. View full document. Students also studied. Unit 3 Homework.docx. Solutions Available. SUNY Empire State College.

  13. ASL Homework 3:9

    Signing Naturally Unit 1-6 This is me just checking course hero. not sure if they are all correct. ASL ... Chapter 22 Homework. 47 terms. zguldeme. Preview. english-moon of the crusted snow. 11 terms. oliviajohnson183. Preview. Blackfoot Numbers 1-10. Teacher 10 terms. Lspottedbull. Preview.

  14. ASL 1

    American Sign Language 1 (CSD 138) 53 Documents. Students shared 53 documents in this course. University Butler University. Academic year: 2022/2023. ... ASL 1 - 2.4 Homework; ASL 1 - 2.3 Homework; ASL 1 - 2.2 Homework; Deaf Out Loud ASL 1 EXAMPLE; ASL Deaf Like Me Reflection EXAMPLE; Related Studylists ASL. Preview text.

  15. Signing Naturally Unit 3.5.pdf

    ASL_Homework_3.5. Badger High School. SCIENCE 30 101` Glass. Primary color. Orange. ASL_Homework_3.5. View ASL_Homework_3.5 from SCIENCE 30 101` at Badger High School. ASL 2 Homework 3.5 Bu... Signing Naturally 3.5.docx. Community College of Denver. ASL 121. Black and white films. Primary color.

  16. Unit 3-5-3-8 (docx)

    [ASL I HOMEWORK] Unit 3:5- 3:8 Name:_____ As you watch the video and follow along in your textbook you will see sections that have spaces that require an answer. Instead of filling in the answers in your book, you will TYPE them into this document and submit it on Blackboard. (If something is to be circled, please type in what you would have circled in the book.)

  17. American Sign Language II (SGNL 1402 34882)

    Course Syllabus. American Sign Language (ASL) Beginning II. SGNL 1402. Semester with Course Reference Number (CRN) Fall 2014, CRN 34882. Instructor contact information (phone number and email address) Lyman (Joseph) Mann. [email protected] or [email protected]. Office Location and Hours.

  18. Signing Naturally Unit 3 Answers

    American Sign Language II (ASL 114) 6 Documents. Students shared 6 documents in this course. University College of Staten Island CUNY. Academic year: 2022/2023. Uploaded by: Anonymous Student. ... Homework 3: Minidialogue 1 - ASL class is three hours. Minidialogue 2 - nap is twenty minutes. Minidialogue 3 - Baking cookies takes 10-12 minutes.

  19. Unit 3 ASL HW.docx

    Unformatted text preview: 1) 65 fish in the bowl 2) 47 classrooms at the nearby high school 3) 31 people learning to fish 4) 54 hills leading up to his house 5) His dad lives in an apartment building with 50 apartments 6) He has ridden the bus to college 66 times 7) 51 ASL books at the library 8) 62 Minidialogues (3.9) 1) Ryan lives nearby with two roommates: 1 man and 1 woman.

  20. ASL Homework 3:12

    Talking to rosa on the phone - 2 hours. Minidialogue 8. QUILTING - 8 HOURS. Minidialogue 9. DOING HOMEWORK - 45 MINUTES. Minidialogue 10. WAITING IN LINE - 7 MINUTES. In these minidialogues, one person asks how long it takes to do an activity. Choose the activity and the amount of time given. here is a video with the sig….

  21. GPT, Claude, Llama? How to tell which AI model is best

    W hen Meta, the parent company of Facebook, announced its latest open-source large language model (LLM) on July 23rd, it claimed that the most powerful version of Llama 3.1 had "state-of-the-art ...

  22. ASL Homework 3:10

    Minidialogue 3. 1. What need is expressed? He needs to find a soda machine to buy orange juice. 2. What information is given? There isn't a soda machine with orange juice, but she can buy some at the restaurant located down the hall, and to the right. What does Joey ask Melinda to do?

  23. ASL Homework 3.5.doc

    View Homework Help - ASL Homework 3.5.doc from ASL 49350 at Kent State University. Name: Imari Menefee Minidialogues p.120-121 Watch the three minidialogues and answer the questions below. Observe

  24. ASL HW3.5.pdf

    ASL 3.5 Homework - Google Docs.pdf. Colorado State University, Global Campus. SPH 251. San Francisco Giants. Orange. ASL 3.5 Homework - Google Docs.pdf. View ASL 3.5 Homework - Google Docs.pdf from SPH 251 at Colorado State University, Glob... 3.5 Minidialogues.pdf. Gallaudet University. ASL & DEAF 211.