为什么这么简单的一个问题,所有不联网的 AI 全部都答错了

7 天前
 drymonfidelia
please sort unity event by their execution order:
Start Awake OnPointerDown OnBeginDrag OnDrag OnEndDrag OnPointerUp

正确的顺序 OnPointerUp 在 OnEndDrag 之前,所有 AI 不联网都按照问题里错误的顺序胡编 OnEndDrag > OnPointerUp ,我测试了好几遍都这样

但是如果把问题改成

please sort unity event by their execution order:
Start Awake OnPointerDown OnBeginDrag OnDrag OnPointerUp OnEndDrag

(用正确的顺序提问)

除了 Claude 3.5 Sonnet 还在坚持 OnEndDrag > OnPointerUp ,还详细胡编了一个解释 其它 AI 都能答对

在 Unity 里不管代码顺序怎么样,OnPointerUp 始终比 OnEndDrag 先调用

另外我还测试了把 OnBeginDrag 写错成 OnDragBegin ,OnEndDrag 写成 OnDragEnd ,只有推特的 Grok 发现了问题,别的 AI 全都按错误的事件名在回复,完全没有怀疑

4095 次点击
所在节点    程序员
35 条回复
akira
7 天前
现在 ai 的 底层就是这么个逻辑呀,知识图谱,检索增强 就是为了解决垂直领域瞎扯淡 的问题提出来的解决方案。
clorischan
7 天前
人脑也是差不多, 可以了解一下对经过裂脑手术(胼胝体切开术, 切断左右脑连接)的人相关研究

人脑的语言中枢在左脑, 如果将信息只输入给左脑,
人的回答也会发生类似目前 LLM 的幻觉现象, 一本正经的胡说八道.
coderluan
7 天前
因为这些模型只是大语言模型,语言之外的东西本身就是他能力之外的,没训练到的东西他都会胡说八道。因为计算机语言也是“语言”,所以已经不错了,让人会误会它已经很智能了。但是你问他点更容易的娱乐业问题,比如让他给你推荐点最新的游戏,答案基本是没眼看的。
marquina
7 天前
大模型只是个工具。任何工具(甚至任何人)都可能在一方面表现不好、在另一方面表现不好(即使这个方面在另一个工具/人看来只是 easy 级别的)。

而使用工具最高效的方式,就是将其应用在它最擅长的方面,而不是质问这个工具为什么做不好它不擅长的方面。
yqcode
7 天前
大模型不是检索知识再输出,而是预测生成,生成出来的答案准确只是因为有了大量的事实训练样本。
StevenRCE0
7 天前
@lscho
@drymonfidelia

人们没有显性强制 LLM 必须回答,这个行为被融合在对齐过程里了,对于它来说是“瞎说和讲我不知道哪种更讨喜”
635925926
7 天前
没啥问题啊,你让小学生来也回答不出啊。
635925926
7 天前
惭愧,我竟然不会 op 口中的“这么简单的一个问题”
zlowly
7 天前
不止是小学生。就算是我们人类自己,即使对很多领域都完全不了解,例如我就对大模型如何生成答案的技术完全不懂,但完全不妨碍我做自己利用其他领域知识或者自己的臆测,做出好像很肯定好像很专业的回答,即使这个回答其实是完全错误的。
riceball
7 天前
提示词没有设计好,因为这个的确算小众知识,如果要激发出来,提示词是关键。

我的测试是,本地 7B 左右的 LLM 基本没戏,激发也没用,已经吃不准了,而 qwen2.5 则没戏,估计是一点都没有学过。
7B 左右只有 gemma9B 经过二次提醒,能搞对顺序。30b 左右的基本能一次搞定。

[PPE(Programable Prompt Engine)]( https://github.com/offline-ai/cli)提示词如下:

```yaml
---
parameters:
temperature: 0
---
system: |-
You are a helpful Unity programming expert in the world. You know the Unity knowledge very well. You are very careful and precise.
Think about the user's intention carefully and solve the problem step by step.
---
user: |-
Please tell me the order in which the following Unity events occur:
OnPointerUp Start Awake OnPointerDown OnBeginDrag OnDrag OnEndDrag
assistant: "[[AI]]"
```

`ai run unity.ai.yaml`的执行结果(gemma-27b):

```
user: Please tell me the order in which the following Unity events occur:
OnPointerUp Start Awake OnPointerDown OnBeginDrag OnDrag OnEndDrag


unity: Here's the order of those Unity events, along with a brief explanation of each:

1. **Awake:** This event is called once when the script instance is being loaded, before any Start functions are called.

2. **Start:** This event is called once at the beginning of the game, after all Awake functions have been called.

3. **OnPointerDown:** This event is triggered when the user starts pressing down on a UI element or a GameObject with a
Collider.

4. **OnBeginDrag:** This event is triggered when the user starts dragging the UI element or GameObject after pressing down.

5. **OnDrag:** This event is triggered continuously while the user is dragging the UI element or GameObject.

6. **OnPointerUp:** This event is triggered when the user releases the mouse button or touch input.

7. **OnEndDrag:** This event is triggered when the user finishes dragging the UI element or GameObject.

**Important Notes:**

* These events are typically used with UI elements or GameObjects that have colliders attached.
* The order can be slightly modified depending on specific user interactions and the setup of your UI or game.



Let me know if you have any other Unity event-related questions!
```

gemma9B 经过二次提醒的提示词:

```yaml
---
parameters:
temperature: 0
---
system: |-
You are a helpful Unity programming expert in the world. You know the Unity knowledge very well. You are very careful and precise.
Think about the user's intention carefully and solve the problem step by step.
---
user: |-
Please tell me the order in which the following Unity events occur:
OnPointerUp Start Awake OnPointerDown OnBeginDrag OnDrag OnEndDrag
assistant: "[[AI]]"
user: |-
Hint: Please think about OnPointerUp in Unity carefully.
assistant: "[[AI]]"
```

执行结果:

```
user: Please tell me the order in which the following Unity events occur:
OnPointerUp Start Awake OnPointerDown OnBeginDrag OnDrag OnEndDrag


unity: Here's the order in which those Unity events occur, along with a brief explanation of each:

1. **Awake:** This method is called once, when the script instance is being loaded. It's used to initialize variables and set up
initial conditions.

1. **Start:** This method is called once, after Awake, and before the first frame update. It's often used to perform setup tasks
that require access to the GameObject's components or other game objects.

1. **OnPointerDown:** This event is triggered when the user presses a mouse button or touches the screen over your GameObject.

2. **OnBeginDrag:** This event is triggered when the user starts dragging an object. This usually happens after `OnPointerDown`
and only if the object is set up to be draggable.

1. **OnDrag:** This event is triggered repeatedly while the user is dragging the object.

2. **OnEndDrag:** This event is triggered when the user releases the mouse button or lifts their finger from the screen, ending
the drag operation.

1. **OnPointerUp:** This event is triggered when the user releases the mouse button or lifts their finger from the screen. It
occurs after `OnEndDrag` if the drag operation was initiated.


**Important Notes:**

* The exact order of events can vary slightly depending on the specific Unity version and the context of the events.
* `OnPointerDown` and `OnPointerUp` are more general events that can occur for any interaction with the mouse or touch input.
`OnBeginDrag`, `OnDrag`, and `OnEndDrag` are specifically related to dragging operations.



Let me know if you have any other questions about Unity events or need further clarification!

user: Hint: Please think about OnPointerUp in Unity carefully.

unity: You're right, I need to be more careful about `OnPointerUp`.

While it's generally triggered when the user releases the mouse button or finger, it's not *always* directly after `OnEndDrag`.

Here's a more precise order, taking `OnPointerUp` into account:

1. **Awake:** Initialization of script variables.
2. **Start:** Setup tasks after Awake.
3. **OnPointerDown:** Mouse button pressed or touch initiated on the GameObject.
4. **OnBeginDrag:** Drag operation starts if the GameObject is set up for dragging.
5. **OnDrag:** Repeatedly triggered while dragging.
6. **OnPointerUp:** Triggered when the mouse button or finger is released.
* **Important:** This can happen *before* `OnEndDrag` if the user releases the button without completing the drag.

7. **OnEndDrag:** Triggered when the drag operation officially ends (usually after `OnPointerUp` if the drag was completed).



Let me know if you have any more questions!
```
drymonfidelia
7 天前
@riceball 但是还是需要把 OnPointerUp 挪到最前面,如果我本来不知道正确的顺序,就不知道 AI 很容易在这个地方弄错
riceball
7 天前
@drymonfidelia OnPointerUp 在最前面就是个错误顺序,放这里也一样,`Start OnPointerUp Awake OnPointerDown OnBeginDrag OnDrag OnEndDrag`。这个提示词思想是让它按事件发生的先后顺序来讲解,因此与你写在里面的顺序无关. 另外如果要让纠正名称错误,最好单独一个提示词,一次只让它做一件事也能提高成功率。
但说实话,我发现它在语法纠错上不是很好,尤其是漏字,可能是没有针对这块精调过。
catamaran
7 天前
我觉得对于 ai 来讲,判断是否知道就是个很难的问题,我们交流用的语言本身就有点玄学。一道数学题,你可以明确回答不知道,一道语文题,比如选择最合适的词语,总结中心思想,没办法说会不会,也保证不了回答的正确性
SoyaDokio
7 天前
@yuzo555 #5
个人完全不懂这些 AI 原理的任何背景知识,但从我个人判断,它们应该还是有本质区别的。
因为它不知道的时候它会胡说八道,而不是说“我不知道”。这存在两种可能:
①它知道它不知道
②它不知道它不知道

个人认为运营 AI 的资本不至于采取中国式商业习惯——不懂装懂,至少不至于全部的资本都这样,如果判断成立,那就是说②是事实,即它并不知道它不知道。

而知道自己不知道,靠的是思考和认知;但如果不理解问题本身而只是鹦鹉学舌的话,当然不知道自己是否知道。
yolee599
6 天前
总有人以为现有的 AI 具有思考能力,甚至以为有计算能力。现有的 AI 其实就是文本生成,一个输入,一个输出,你可以输入任何东西,输出也有可能是任何东西。就像 AES 解密,密钥不对它也会输出结果

这是一个专为移动设备优化的页面(即为了让你能够在 Google 搜索结果里秒开这个页面),如果你希望参与 V2EX 社区的讨论,你可以继续到 V2EX 上打开本讨论主题的完整版本。

https://www.v2ex.com/t/1101072

V2EX 是创意工作者们的社区,是一个分享自己正在做的有趣事物、交流想法,可以遇见新朋友甚至新机会的地方。

V2EX is a community of developers, designers and creative people.

© 2021 V2EX