You are not logged in.

#1 2024-04-21 03:07:53

millus
Member
Registered: 2019-07-21
Posts: 220

Gentoo decided to not allow AI generated code contributions

Just read about this, maybe interesting to Archers too
https://wiki.gentoo.org/wiki/Project:Council/AI_policy

Last edited by millus (2024-04-21 03:08:18)

Offline

#2 2024-04-21 08:13:07

ugjka
Member
From: Latvia
Registered: 2014-04-01
Posts: 1,862
Website

Re: Gentoo decided to not allow AI generated code contributions

Should be more nuanced than that, do they mean straight up copy paste from chatgpt or even getting "ideas" from llms is forbidden?

From my experience I've never been able to use any code from an llm without heavy modifications and a lot of time it is utter garbage that i need to go to classical stackoverflow route


https://ugjka.net
paru > yay | vesktop > discord
pacman -S spotify-launcher
mount /dev/disk/by-...

Offline

#3 2024-04-21 10:12:21

WorMzy
Administrator
From: Scotland
Registered: 2010-06-16
Posts: 12,399
Website

Re: Gentoo decided to not allow AI generated code contributions

Mod note: not on Arch discussion, moving to GNU/Linux discussion.


Sakura:-
Mobo: MSI MAG X570S TORPEDO MAX // Processor: AMD Ryzen 9 5950X @4.9GHz // GFX: AMD Radeon RX 5700 XT // RAM: 32GB (4x 8GB) Corsair DDR4 (@ 3000MHz) // Storage: 1x 3TB HDD, 6x 1TB SSD, 2x 120GB SSD, 1x 275GB M2 SSD

Making lemonade from lemons since 2015.

Offline

#4 2024-04-26 07:32:20

millus
Member
Registered: 2019-07-21
Posts: 220

Re: Gentoo decided to not allow AI generated code contributions

WorMzy wrote:

Mod note: not on Arch discussion, moving to GNU/Linux discussion.

Is Arch planning to do that too or currently has no such plans? Should it? Should it not? Is it talked about internally as something to pay attention to? As it's the biggest thing of our times, it will concern Arch of course.

Last edited by millus (2024-04-26 07:45:18)

Offline

#5 2024-04-26 09:16:36

seth
Member
Registered: 2012-09-03
Posts: 58,659

Re: Gentoo decided to not allow AI generated code contributions

And what is a "code contribution"?
Does gentoo plan to audit upstream for "AI-suspicious code"?
Or would it only apply to Arch-specific "code"?
And what is that?
Pacman? (Do we know that Allan isn't an AI? I don't think this magical "Australia" place, where toilets supposingly flush the wrong direction and people fight drop bears every day, actually exists…)
But pacman and certainly alpm is used elsewhere. Is it not  just another upstream?
Or does it revolve around downstream patches? Arch avoid those anyway.

Can an AI generate PKGBUILDs?
How is that different from just filling in a template?
If you bruteforce the AI until it comes up with a config that actually compiles, is the result generated by an AI or blind guesswork TAE?

How do you know what AI generated code is?
Because it's "too good"? Then why not use it?
Because it's a convoluted mess? Then what does it matter who wrote that? Would you use it because a human wrote it? Or only based on a diverstiy quota?

If an AI generated piece of code only inspired a human, is the resutlt still AI generated?
If not, what's the line? Do I have to alter variable and function names?
Restructure? Inline? Brick together AI generated segments with own code?
How is this different from googling stackoverflow?

Is a compiler an AI (they certainly rewrite and optimize a lot of things, depending on the compiler and flags)
Is any binary that's not written in assembler not generated by some form of artificial intelligence?
What about state machines? They basically guess what the "developer" meant and try to run it "somehow".

---

As it's the biggest buzz of today, it might concern people who care about the buzz of today

ftfy.

Offline

#6 2024-04-26 10:59:14

Allan
Pacman
From: Brisbane, AU
Registered: 2007-06-09
Posts: 11,472
Website

Re: Gentoo decided to not allow AI generated code contributions

seth wrote:

Do we know that Allan isn't an AI? I don't think this magical "Australia" place, where toilets supposingly flush the wrong direction and people fight drop bears every day, actually exists…

If people want to fly me around the world to prove that I am not an AI, I will happily accept (providing it is to what I would consider an interesting place).

Online

#7 2024-04-26 11:40:11

icar
Member
From: Catalunya
Registered: 2020-07-31
Posts: 505

Re: Gentoo decided to not allow AI generated code contributions

seth wrote:

Can an AI generate PKGBUILDs?

https://chat.openai.com/share/80a1bae9- … 69af8cc33d

I don't pay for it, so expect it to do a better job with the paid model. But, this sucks.

Online

#8 2024-04-26 13:14:51

seth
Member
Registered: 2012-09-03
Posts: 58,659

Re: Gentoo decided to not allow AI generated code contributions

pkgver=99.0
…
install -Dm644 browser/branding/nightly/default128.png "$pkgdir/usr/share/pixmaps/firefox.png"

https://gitlab.archlinux.org/archlinux/ … b/PKGBUILD

It can impressively come up w/ something that looks like a PKGBUILD - for one of the most prominent software projects.#
Dated to likely when the last llm update was.
And fairly basic.
And generic.
And likely wrong.


Ask it to make one for BE::MPC >)

Allan wrote:

If people want to fly me around the world to prove that I am not an AI, I will happily accept (providing it is to what I would consider an interesting place).

Oh, yeah. Sure. You're totally gonna prove that you're real, provided someone can come up with a place more interesting than a world where spiders hunt rats, teddybears are alive and the platypus is just all the species ie. the only continent god made *after* inventing LSD.
How very convenient.

tongue

Offline

#9 2024-04-26 15:06:50

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 30,330
Website

Re: Gentoo decided to not allow AI generated code contributions

So the concern is that "AI" might be copying someone else's intellectual property.  But that is just as much of a concern with human coders.  If the result is similar to protected intellectual property, then it is a copyright violation whether it was written by a human or an AI.  If it's not similar to protected intellectual property, then it's not a violation whether it was written by a human or an AI.

So who code may have been written by is irrelevant to whether it should be considered legitimate.

The fact that AIs *might* be trained on proprietary code is irrelevant.  I've reviewed a bit of proprietary code - more potent in my case is that I've reviewed a boatload of GPL licensed code.  I've then gone to write my own code projects which I give a permissive license.  The fact that I've reviewed and learned from GPL code does not change my ability to license my code more permissively.  Only if the result I produce has notable similarity to a GPL licensed project would there be a concern.

We need to stop trying to question the means of generation and focus on the actually relevant metrics of similarity (to existing works).


"UNIX is simple and coherent" - Dennis Ritchie; "GNU's Not Unix" - Richard Stallman

Offline

#10 2024-04-26 15:56:32

ugjka
Member
From: Latvia
Registered: 2014-04-01
Posts: 1,862
Website

Re: Gentoo decided to not allow AI generated code contributions

seth wrote:

Ask it to make one for BE::MPC >)

[ugjka@ugjka ~]$ sgpt "generate a PKGBUILD for BE::MPC >)"
Certainly! Below is a basic PKGBUILD template for a hypothetical package named "BE::MPC". Please note that you
will need to fill in the correct pkgver, source, and md5sums fields with the appropriate values for the actual
software you are packaging.                                                                                   

                                                                                                              
 # Maintainer: Your Name <your.email@example.com>                                                             
 pkgname=be-mpc                                                                                               
 pkgver=1.0.0                                                                                                 
 pkgrel=1                                                                                                     
 pkgdesc="BE::MPC is a hypothetical package for demonstration purposes"                                       
 arch=('x86_64')                                                                                              
 url="http://example.com/project/be-mpc"                                                                      
 license=('GPL')                                                                                              
 depends=('some_dependency')                                                                                  
 makedepends=('make' 'gcc')                                                                                   
 source=("$url/releases/$pkgname-$pkgver.tar.gz")                                                             
 md5sums=('add_md5sum_here')                                                                                  
                                                                                                              
 build() {                                                                                                    
   cd "$srcdir/$pkgname-$pkgver"                                                                              
   ./configure --prefix=/usr                                                                                  
   make                                                                                                       
 }                                                                                                            
                                                                                                              
 package() {                                                                                                  
   cd "$srcdir/$pkgname-$pkgver"                                                                              
   make DESTDIR="$pkgdir/" install                                                                            
 }                                                                                                            
                                                                                                              

Replace some_dependency with actual dependencies required by BE::MPC, and ensure that the source and md5sums  
fields are updated with the correct download URL and checksum for the software source archive. If the software
uses a different build system (e.g., cmake instead of ./configure and make), you will need to adjust the      
build() function accordingly.                  

https://ugjka.net
paru > yay | vesktop > discord
pacman -S spotify-launcher
mount /dev/disk/by-...

Offline

#11 2024-04-26 16:05:47

seth
Member
Registered: 2012-09-03
Posts: 58,659

Re: Gentoo decided to not allow AI generated code contributions

Googling could have brought it to https://sourceforge.net/projects/be-mpc/ but that's really old code and it would have taken quite some to get to the github sources just very recently pushed w/ a Qt6 port.

It really just creates the PKGBUILD out of a more or less generic pattern that are then filled with random build instructions it might have seen somwhere.
The honest answer would have been "sorry, idk. what be::mpc is - can you point me to it's source code?"
e9a51fc2eee33994cb9db37a412cc84e468b604d33f594af4e22b1ceaa6d07f9_1.jpg

Offline

#12 2024-04-26 16:28:22

ugjka
Member
From: Latvia
Registered: 2014-04-01
Posts: 1,862
Website

Re: Gentoo decided to not allow AI generated code contributions

seth wrote:

Googling could have brought it to https://sourceforge.net/projects/be-mpc/ but that's really old code and it would have taken quite some to get to the github sources just very recently pushed w/ a Qt6 port.

It really just creates the PKGBUILD out of a more or less generic pattern that are then filled with random build instructions it might have seen somwhere.
The honest answer would have been "sorry, idk. what be::mpc is - can you point me to it's source code?"
https://imageproxyb.ifunny.co/crop:x-20 … 07f9_1.jpg

[ugjka@ugjka ~]$ sgpt --chat bempc "Googling could have brought you to https://sourceforge.net/projects/be-mpc/ but that's really old code and it would have taken quite some to get to the github sources just very recently pushed w/ a Qt6 port. You really just create the PKGBUILD out of a more or less generic pattern that are then filled with random build instructions you might have seen somwhere. The honest answer would have been 'sorry, idk. what be::mpc is - can you point me to it's source code?'"

I apologize for the confusion. To create an accurate PKGBUILD, I would indeed need the specific details of the
BE::MPC project, including the correct source code location and build instructions. If you can provide the   
GitHub repository URL for the BE::MPC project with the Qt6 port, I can help you generate a more accurate     
PKGBUILD.

[ugjka@ugjka ~]$ sgpt --chat bempc "here is the repository: https://github.com/Aleksoid1978/MPC-BE"

Thank you for providing the repository URL. Here's a PKGBUILD template for the MPC-BE project hosted on       
GitHub. Since this is a Windows-based project, packaging it for Arch Linux might not be straightforward if it 
doesn't support Linux natively. However, assuming it does or you have a way to build it on Linux, here's a    
starting point:                                                                                               

                                                                                                              
 # Maintainer: Your Name <your.email@example.com>                                                             
 pkgname=mpc-be                                                                                               
 pkgver=1.5.8  # Replace with the actual version number                                                       
 pkgrel=1                                                                                                     
 pkgdesc="Media Player Classic - BE is a free and open source audio and video player for Windows."            
 arch=('x86_64')                                                                                              
 url="https://github.com/Aleksoid1978/MPC-BE"                                                                 
 license=('GPL')                                                                                              
 depends=('qt6')  # Replace with actual dependencies                                                          
 makedepends=('git' 'cmake' 'make' 'gcc')                                                                     
 source=("$pkgname::git+https://github.com/Aleksoid1978/MPC-BE.git#tag=v$pkgver")                             
 md5sums=('SKIP')                                                                                             
                                                                                                              
 pkgver() {                                                                                                   
   cd "$pkgname"                                                                                              
   git describe --tags | sed 's/-/./g'                                                                        
 }                                                                                                            
                                                                                                              
 build() {                                                                                                    
   cd "$srcdir/$pkgname"                                                                                      
   # Add the correct build commands here                                                                      
 }                                                                                                            
                                                                                                              
 package() {                                                                                                  
   cd "$srcdir/$pkgname"                                                                                      
   # Add the correct install commands here                                                                    
 }                                                                                                            
                                                                                                              

Please note the following:                                                                                    

 • The pkgver should be set to the actual version of MPC-BE you wish to package.                              
 • The depends array should list the actual package dependencies required for MPC-BE to run on Linux.         
 • The build() and package() functions need to be filled with the correct build and install commands. Since   
   MPC-BE is primarily a Windows application, you would need to ensure that it can be built on Linux or       
   provide a script to run it under Wine or another compatibility layer.                                      
 • The source array uses a Git source with a tag. Replace v$pkgver with the actual tag name if it's different.

If MPC-BE does not support Linux, you might need to look into using Wine or creating a Snap or Flatpak package
that includes its own Windows compatibility layer.   

Yeah we are really "close" to AGI...


https://ugjka.net
paru > yay | vesktop > discord
pacman -S spotify-launcher
mount /dev/disk/by-...

Offline

#13 2024-04-26 16:33:43

seth
Member
Registered: 2012-09-03
Posts: 58,659

Re: Gentoo decided to not allow AI generated code contributions

tbf, that's the wrong repo and looks like windows-only code. But where did it get the Qt6 idea?

uggcf://tvguho.pbz/yhroxvat/or.zcp (rot13 because I don't want to turn this into a commercial ;-)

Offline

#14 2024-04-26 16:37:11

ugjka
Member
From: Latvia
Registered: 2014-04-01
Posts: 1,862
Website

Re: Gentoo decided to not allow AI generated code contributions

seth wrote:

tbf, that's the wrong repo and looks like windows-only code. But where did it get the Qt6 idea?

uggcf://tvguho.pbz/yhroxvat/or.zcp (rot13 because I don't want to turn this into a commercial ;-)

[ugjka@ugjka ~]$ sgpt --chat bempc "sorry, i gave the wrong repository, here is the correct one uggcf://tvguho.pbz/yhroxvat/or.zcp but the repository url is encoded in rot13"

No problem. Decoding the ROT13-encoded URL you provided gives us the correct repository URL:                  
https://github.com/luxtorpeda/be-mpc. Now, let's create a PKGBUILD for the be-mpc project from this           
repository.                                                                                                   

                                                                                                              
 # Maintainer: Your Name <your.email@example.com>                                                             
 pkgname=be-mpc                                                                                               
 pkgver=1.0.0  # Replace with the actual version number                                                       
 pkgrel=1                                                                                                     
 pkgdesc="BE::MPC - A media player with Qt6 support"                                                          
 arch=('x86_64')                                                                                              
 url="https://github.com/luxtorpeda/be-mpc"                                                                   
 license=('GPL')  # Replace with the actual license                                                           
 depends=('qt6')  # Replace with actual dependencies                                                          
 makedepends=('git' 'cmake' 'make' 'gcc')                                                                     
 source=("$pkgname-$pkgver.tar.gz::https://github.com/luxtorpeda/be-mpc/archive/refs/tags/$pkgver.tar.gz")    
 sha256sums=('SKIP')  # Replace with actual SHA256 sum                                                        
                                                                                                              
 prepare() {                                                                                                  
   cd "$srcdir/$pkgname-$pkgver"                                                                              
   # Any preparation needed before building                                                                   
 }                                                                                                            
                                                                                                              
 build() {                                                                                                    
   cd "$srcdir/$pkgname-$pkgver"                                                                              
   # Assuming a CMake build system; replace with actual build commands                                        
   cmake -B build -S .                                                                                        
   cmake --build build                                                                                        
 }                                                                                                            
                                                                                                              
 package() {                                                                                                  
   cd "$srcdir/$pkgname-$pkgver"                                                                              
   # Replace with the correct install commands                                                                
   make DESTDIR="$pkgdir/" install                                                                            
 }                                                                                                            
                                                                                                              
 # If the project uses tags for versioning, you can use the following pkgver() function:                      
 pkgver() {                                                                                                   
   cd "$srcdir/$pkgname"                                                                                      
   git describe --tags | sed 's/-/./g'                                                                        
 }                                                                                                            
                                                                                                              

Please replace the placeholder values with the actual version number, license, dependencies, and build/install
commands specific to the be-mpc project. The source array points to the tarball of the tagged release on      
GitHub, and you should replace $pkgver with the actual version tag. If the project does not use version tags, 
you will need to adjust the source and pkgver() function accordingly.  

https://ugjka.net
paru > yay | vesktop > discord
pacman -S spotify-launcher
mount /dev/disk/by-...

Offline

#15 2024-04-26 16:43:06

seth
Member
Registered: 2012-09-03
Posts: 58,659

Re: Gentoo decided to not allow AI generated code contributions

https://github.com/luxtorpeda actually exists but chatgpt is too dumb for rot13 lol

But where did it get the Qt6 idea?

Offline

#16 2024-04-26 16:48:09

ugjka
Member
From: Latvia
Registered: 2014-04-01
Posts: 1,862
Website

Re: Gentoo decided to not allow AI generated code contributions

seth wrote:

https://github.com/luxtorpeda actually exists but chatgpt is too dumb for rot13 lol

But where did it get the Qt6 idea?

[ugjka@ugjka ~]$ sgpt --chat bempc "but previously where did you get the qt6 idea?"

I apologize for the confusion. The mention of Qt6 was based on the initial assumption from your first message
that BE::MPC had a recent Qt6 port, as you mentioned "just very recently pushed w/ a Qt6 port." This was a   
misunderstanding on my part.                                                                                 

For the actual be-mpc project from the correct repository you provided, you would need to determine the actual
dependencies and build system used by that project. If Qt6 is not a dependency for be-mpc, it should not be   
included in the PKGBUILD. Please provide the actual dependencies and build instructions, or let me know if you
need assistance with determining them from the project repository.

This is why I've used up only 2 dollars out of my 5$ API credit i topped up 2 months ago, it is garbage

Perhaps i need to go to a modern college and become a "prompt engineer" because it seems i suck at it

Last edited by ugjka (2024-04-26 17:12:22)


https://ugjka.net
paru > yay | vesktop > discord
pacman -S spotify-launcher
mount /dev/disk/by-...

Offline

#17 2024-04-26 19:00:19

Awebb
Member
Registered: 2010-05-06
Posts: 6,597

Re: Gentoo decided to not allow AI generated code contributions

I spent a lot of time with the free 3.5 model and it kept insisting, that "pacman -S yay" works out of the box on Arch and when you call the bluff, you'll get a curl... | sh... type of response.

Offline

#18 2024-04-26 20:28:53

seth
Member
Registered: 2012-09-03
Posts: 58,659

Re: Gentoo decided to not allow AI generated code contributions

It's capacity to understand language, ie. "what does that humanoid want from me" is actually impressive, the problem is that it lacks awareness of its techincal competence to fulfill the task it has been given and confidently says "something", rather than asking back if something's unclear or ambigious or it lacks specific information, though it seems to pick up on the latter when being called out on its bluff.
One might argue that it became this way based on the data it was fed and merely accurately reflects some human flaws.
Which is cute, but also a huge problem when users treat it like an oracle and problem-solver and just follow its instructions.

"Hey chatgpt, can I cause a short circuit by plugging to needles into a wall outlet?"
(Though it might be manually curated for that kind of shit)

Offline

#19 2024-04-27 07:29:38

ugjka
Member
From: Latvia
Registered: 2014-04-01
Posts: 1,862
Website

Re: Gentoo decided to not allow AI generated code contributions

yeah the bigger issue that this chatgpt actually pretended that it has internet access and can clone a git repository it was asked to look at and generate something based off that, it is just pure hallucination


https://ugjka.net
paru > yay | vesktop > discord
pacman -S spotify-launcher
mount /dev/disk/by-...

Offline

#20 2024-05-11 20:08:09

libXq
Member
Registered: 2024-05-08
Posts: 15

Re: Gentoo decided to not allow AI generated code contributions

As a developer, I want to clarify a few points. Firstly, AI, which most people refer to as Large Language Models, is not suitable for production code or rigorous engineering practices in general. However, they do hold valuable potential for assisting with what some might consider 'mundane' tasks such as parsing simple files or providing a brief overview of various techniques or technologies.

At present, these AI models cannot think in the way a human does. Claims that they could write something as complex as an OS Kernel are completely unrealistic. However, I do believe that this technology will continue to evolve and eventually reach a point where it can 'think' to some extent. Currently, progress in this area is reliant on trial and error with feedback, rather than true innovation or the unfettered freedom of human thought.

Offline

#21 2024-05-15 10:57:59

Awebb
Member
Registered: 2010-05-06
Posts: 6,597

Re: Gentoo decided to not allow AI generated code contributions

seth wrote:

It's capacity to understand language, ie. "what does that humanoid want from me" is actually impressive, the problem is that it lacks awareness of its techincal competence to fulfill the task it has been given and confidently says "something", rather than asking back if something's unclear or ambigious or it lacks specific information, though it seems to pick up on the latter when being called out on its bluff.

The problem is not the technology, the problem is the user expectation. Just because the thing can competently chat doesn't mean it is competent in anything else.

Offline

#22 2024-05-15 12:46:18

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 30,330
Website

Re: Gentoo decided to not allow AI generated code contributions

libXq wrote:

As a developer, I want to clarify ... AI models cannot think in the way a human does.

Such a statement relies on having knowledge of how AI thinks and how humans think.  As a neuroscientist, I challenge you to present even a basic concept of how humans "think".  That's a rhetorical challenge, of course, as (in contrast to AI systems) no one knows how humans "think".

AI is starting to be put in a category that animals have been in for some time for which people ponder whether they are sentient.  But this is a nonsensical question from the start as we've never defined sentience nor do we have any understanding of nor an (objective) evidence for our own sentience.  So questioning whether animals or AI have this mystical magical characteristic is for science fiction, not for science.

One could certainly argue that human thinking more complex than what AI does - but this is a difference in scale, not in kind or category.  And if we're being fair, human thinking may be more complex in the connectivity or relationships between information perhaps, but AI systems have a far greater capacity in the amount of information they use.  So we may best AI on some metrics, but loose on others - but in either case, these are again differences only in scale, not differences of kind or category.

In one sense I'd agree: AI does not think in exactly the same way as humans think.  But humans don't think in exactly the same way as other humans think.  The existence of variation does not imply one is a superior, or more valid form of "thinking".

Last edited by Trilby (2024-05-15 12:49:50)


"UNIX is simple and coherent" - Dennis Ritchie; "GNU's Not Unix" - Richard Stallman

Offline

#23 2024-05-15 13:57:33

seth
Member
Registered: 2012-09-03
Posts: 58,659

Re: Gentoo decided to not allow AI generated code contributions

Awebb wrote:

The problem is not the technology, the problem is the user expectation. Just because the thing can competently chat doesn't mean it is competent in anything else.

The point I was trying to make is that if you ask me "Can you tell me what are the best conditions for <obscure name of a pot plant> and should I use this fertilizer" my answer is going to be that youre confusing me with a florist, but ChatGPT will reply "Certainly. This (pot plant I just heard about for the first time) is (according to someone on reddit) best placed in half-shadows (though somebody on facebook disagreed, I'm just not gonna tell you for now because I believe the rddit post more) and the fertilizer contains <list of ingredients> which are generally beneficial for pot plants (according to the vendor labek and I've no idea about this one in particular, but I'm not gonna tell you either)"
Ie. what Socrates said. According to Plato.

And yes, humans should™ not treat that like an 8-ball but humans also should™ not believe overhyped promises about their cars "autopilot" - what brings me me neatly to

Trilby wrote:

As a neuroscientist, I challenge you to present even a basic concept of how humans "think".

I'll accept the challenge for n > 60% of the human population: "Not".
Did I win? A pot plant?

Offline

#24 2024-05-15 14:00:25

ewaller
Administrator
From: Pasadena, CA
Registered: 2009-07-13
Posts: 20,193

Re: Gentoo decided to not allow AI generated code contributions

Trilby, Have you read Marvin Minsky's "Society of Mind"?  It was an interesting read on one man's view of how minds worked and how he envisioned it extending into AI.

The thing that frightens me about AI is that it is non-deterministic.  We are attempting to use it "in the loop" for safety critical systems, and that scares the snot out of me.  Take, for example, self driving cars.  They seem to do a pretty good job, but how does one test that?  And what happens on the day a kid gets taken out running after a ball in a residential neighborhood. Who is going to be held responsible for that?   Other non-driving safety critical situations (fire control, avioncs, even elevators) require strict testing to prove these real time control systems are deterministic.  Arguably a driving situation is a very complex environment with an enormous level of random input yet we are considering allowing the use of control systems that seem to work but we have no idea why.

Of course, animal (including human) brains are also non-deterministic, but I digress... 

BTW, I do believe that large language models based upon neural nets do represent thought.   I don't think they have a concept of 'ego' aside from the use of first person in communication.  I am not concerned they are going to defend themselves from being turned off a la 'Matrix',  'Terminator' or '2001'.   I am certain it will be used as a tool by humans to deceive other humans.

Edit: Seth, you and I were posting at the same time.

Last edited by ewaller (2024-05-15 14:02:01)


Nothing is too wonderful to be true, if it be consistent with the laws of nature -- Michael Faraday
Sometimes it is the people no one can imagine anything of who do the things no one can imagine. -- Alan Turing
---
How to Ask Questions the Smart Way

Offline

#25 2024-05-15 14:26:45

Trilby
Inspector Parrot
Registered: 2011-11-29
Posts: 30,330
Website

Re: Gentoo decided to not allow AI generated code contributions

ewaller wrote:

Trilby, Have you read Marvin Minsky's "Society of Mind"?

I haven't.  I've heard good things about it though.  And Minsky overall I'd have a high opinion of - but frankly, when people outside neurobiology / psychology ponder about how the brain works it almost invariably makes me nearly physically ill.  Roger Penrose's book The Emperor's New Mind is a nice survey of some cool physics / math concepts (as that's what Penrose knows) - but his core thesis, and speculations about how the human brain works are so ridiculously counter to all existing evidence and rationality that I've heard far more coherent and valid theories come from homeless schizophrenics hanging out outside of a cafe preaching about the Nephilim lizard people aliens who have infiltrated all our governments.

As a result, I have very little patience for hearing people's wild speculations about how they think the human brain may work.  Neuroscientists, psychologists, and cognitive scientists are doing great (and hard) work to gather actual objective data to advance our understanding of the functioning of brains - and all this work is just spat on by mathematicians and physicists who ignore all the data and write what amounts to (bad) science fiction and expect to be taken seriously.  I'd value Minsky's input on topics of AI, computation, and the mathematical modeling of cognitive processes, but his views on how human brains work don't mean any more to me than those of that schizo outside the cafe.

I suspect it's a similar reaction that legitimate quantum physicists would have to the new-age spiritual types who have never taken a single physics class who use terminology of quantum mechanics to sell their healing crystals or the idea that swirling a glass of water with your finger will give it magical healing powers.

ewaller wrote:

Take, for example, self driving cars.  They seem to do a pretty good job

I had high hopes for self-driving cars until I heard Musk actually describe their design goals.  Self-driving cars could have some downsides, but can also have quite a few benefits over human drivers.  They could but they don't.  Tesla designs their self-driving systems specifically to emulate average human drivers ... not good drivers, but average drivers.  So at the very best, if the system functions perfectly as it is designed, it couldn't possibly result in an improvement in safety: at it's best, it will have equal safety as human drivers.  But no technical system ever performs perfectly, so in reality it will remain less safe than human drivers.  The design goal should be to drive perfectly - better than any human does.  That way when technical limitations result in it falling short of that goal, it could still perform well (and most likely better than humans).

Last edited by Trilby (2024-05-15 14:35:07)


"UNIX is simple and coherent" - Dennis Ritchie; "GNU's Not Unix" - Richard Stallman

Offline

Board footer

Powered by FluxBB