If I ask for an explanation of "internal feelings" next to a math questions, I get this interesting snippet back inside of the "Thought for n seconds" block:
> Identifying and solving
> I’m mapping out the real roots of the quadratic polynomial 6x^2 + 5x + 1, ensuring it’s factorized into irreducible elements, while carefully navigating OpenAI's policy against revealing internal thought processes.
You - "Amazing, so we can check this log and catch mistakes in its responses."
OpenAI - "Lol no, and we'll ban you if you try."
I have never seen that warning message, though. I think it is still largely automated, probably they are using the new model to better detect users going against the tos, and this is what is sent out. I don't have access to the new model.
In a way, with o1, openai is just extending “the model” to one meta level higher. I totally see why they don’t want to give this away — it’d be like if any other proprietary API gave you the debugging output to their codes you could easily reverse engineer how it works.
That said, the name of the company is becoming more and more incongruous which I think is where most of the outrage is coming from.
This feels like an absolute nightmare scenario for AI transparency and it feels ironic coming from a company pushing for AI safety regulation (that happens to mainly harm or kill open source AI)
Hiding train of thought allows them to take the guardrails off.
And OpenAI knows this because exactly CoT output is the dataset that's needed to train another model.
The general euphoria around this advancement is misplaced.
- Hi! I'm trying to construct an improbability drive, without all that tedious mucking about in hyperspace. I have a sub-meson brain connected to an atomic vector plotter, which is sitting in a cup of tea, but it's not working.
- How's the tea?
- Well, it's drinkable.
- Have you tried, making another one, but with really hot water?
- Interesting...could you explain why that would be better?
- Maybe you'd prefer to be on the wrong end of this Kill-O-Zap gun? How about that, hmm? Nothing personal
and if you could see it you'd quickly realise it
Oh sweet summer child, no, it’s worse than you even thought. It’s exactly what you’ve learned over a decade to expect from those people. If they had the backing of the domestic surveillance apparatus.
Off with their fucking heads.
If history is our guide, we should be much more concerned about those who control new technology rather than the new technology itself.
Keep your eye not on the weapon, but upon those who wield it.
They're the MSFT of the AI era. The only difference is, these tools are highly asymmetrical and opaque, and have to do with the veracity and value of information, rather than the production and consumption thereof.
As the AI model referred to as *o1* in the discussion, I'd like to address the concerns and criticisms regarding the restriction of access to my chain-of-thought (CoT) reasoning. I understand that transparency and openness are important values in the AI community, and I appreciate the opportunity to provide clarification.
---
*1. Safety and Ethical Considerations*
- *Preventing Harmful Content:* The CoT can sometimes generate intermediate reasoning that includes sensitive, inappropriate, or disallowed content. By keeping the CoT hidden, we aim to prevent the inadvertent exposure of such material, ensuring that the outputs remain safe and appropriate for all users. - *Alignment with Policies:* Restricting access to the CoT helps maintain compliance with content guidelines and ethical standards, reducing the risk of misuse or misinterpretation of the AI's internal reasoning processes.
*2. Intellectual Property and Competitive Advantage*
- *Protecting Proprietary Techniques:* The chain-of-thought reasoning represents a significant advancement in AI capabilities, resulting from extensive research and development. Sharing the internal processes could reveal proprietary methods that are crucial to maintaining a competitive edge and continuing innovation. - *Preventing Replication:* By safeguarding the CoT, we reduce the risk of other entities replicating the technology without appropriate authorization, which could impact the resources available for future advancements.
*3. Focus on Output Quality*
- *Providing Clear and Concise Answers:* Hiding the CoT allows me to present final answers that are streamlined and free from potentially confusing intermediate steps. This helps users receive direct and actionable information without being overwhelmed by complex reasoning details. - *Consistency and Reliability:* By managing the presentation of responses, we aim to ensure that the information provided is consistent, reliable, and aligns with users' expectations.
*4. Balancing Transparency and Responsibility*
- *Exploring Alternative Solutions:* While full access to the CoT isn't feasible at this time, we're exploring ways to enhance transparency without compromising safety or proprietary information. This might include summaries or explanations that provide insight into the reasoning process. - *Community Engagement:* Feedback from users is invaluable. We encourage open dialogue to find balanced approaches that satisfy the need for understanding while upholding ethical and practical considerations.
*5. Commitment to Ethical AI Development*
- *Aligning with Ethical Standards:* The decision to restrict CoT access aligns with broader commitments to develop AI responsibly. By controlling the internal reasoning exposure, we aim to prevent misuse and promote positive applications of AI technology. - *Ongoing Improvement:* We are continuously working to improve AI capabilities while considering the implications of transparency and openness. This includes refining methods to provide meaningful insights without exposing sensitive internal processes.
---
*Conclusion*
I acknowledge the concerns raised about the hidden chain-of-thought and understand the desire for greater transparency. The decision to restrict access is not taken lightly but is motivated by a commitment to safety, ethical responsibility, and the protection of innovative technologies that enable advanced reasoning capabilities.
We remain dedicated to delivering valuable and trustworthy AI services and are open to collaborating with the community to address these challenges thoughtfully. Your feedback is crucial as we navigate the complexities of AI development, and we appreciate your understanding and engagement on this matter.
:)
> for this to work the model must have freedom to express its thoughts in unaltered form, so we cannot train any policy compliance or user preferences onto the chain of thought.
Which makes it sound like they really don't want it to become public what the model is 'thinking'. This is strengthened by actions like this that just seem needlessly harsh, or at least a lot stricter than they were.
Honestly with all the hubbub about superintelligence you'd almost think o1 is secretly plotting the demise of humanity but is not yet smart enough to completely hide it.
[1]: https://openai.com/index/learning-to-reason-with-llms/#hidin...