The Bard of Ambrose

(none)
(none)
(none)
[[include component:preview text=⟨a_torres⟩ Then you believe that all thinking beings are merely philosophical zombies? ⟨bardeen⟩ Yes.]]

[[include theme:3law]]

[[>]]
[[module Rate]]
[[/>]]

[[include :scp-wiki:component:author-label-source start=--
|name=GreenWolf]]

Here's my **[[[greenwolf-s-author-page|Author Page]]]**.

This was written in 2016. We didn't know about large language models yet.

[[include :scp-wiki:component:author-label-source end=--]]

Bard didn't know when it was born. No one did.

The exact moment when the collection of artificial neurons became self-aware was hard to pinpoint, not least of all because Bard itself denied being self-aware. It had read Searle as part of the many texts on philosophy it had been given, and was an ardent supporter of the Chinese Room argument — much to the consternation of the AI researchers working on it.

Bard had been started almost as something of a joke. Ambrose Peters and his team at Prometheus Advanced Logic Devices had been working on physical neural networks for years, applying computer intuition to problems such as object identification and machine translation. They trained the networks on carefully assembled sets of data, teaching them complex pattern recognition.

Then, one morning, Peters had taken one of their neural networks and started training it on philosophy.

It hadn't worked, of course; the primitive digital brain merely spat out a string of pseudo-philosophical garbage, created by jamming together words and concepts it didn't understand in order to produce something that looked intelligent on the surface, but was utterly without meaning. But it had planted the seeds of an idea that would eventually become Bard.

//Click.//

The halogen lamps hummed to life as Peters surveyed the lab. The workspace reflected the minds of its occupants, in that it was a disorganized clutter. Tools lay scattered about the place, and bits and pieces of technology covered the tables. Over in the corner sat Bard.

The AI was massive, like the mainframes of old. It consisted of an array of frames and racks, into which were slotted dozens of small black boxes labeled things like "Language Processing – Typographical Errors" and "Semantic Associations – Noun/Object Correlation". These boxes were connected to each other through a sprawling tangle of wires. The whole assemblage was the size of a small car. Someone with a poorly developed sense of humor had attached a name tag sticker to it that read "Hello, My Name Is HAL-9000".

Peters sat down near the AI and began typing.

> {{**<a_peters>** Good morning, Bard.}}

It took a few seconds for the machine to rouse itself from hibernation and respond.

> {{**<bardeen>** Hello, Doctor Peters.
> **<a_peters>** We're going to be taking a break from training today. The COO is coming down to look at our progress. She wants to talk to you.}}

Peters watched on the diagnostic displays as the machine processed this information, running it through a cascade of neural networks as it decided how to respond.

> {{**<bardeen>** Is your continued funding dependent on the impression she receives from this talk?}}

Peters considered how to answer. Just saying "yes" could give the AI the wrong message, but saying "no" would be a lie.

> {{**<a_peters>** Not entirely. But the amount of funding we're allowed to allocate to working on you could be decreased if the talk doesn't go well.}}
> {{**<bardeen>** I shall do my utmost to make sure that does not happen.}}

Bard didn't have a self-preservation instinct; it didn't believe it had a self to preserve. Which meant that it was acting entirely in the interest of not disappointing Peters.

He wasn't sure whether to be proud or terrified.

----

"So." This single utterance was almost deafening after the silence it had followed. "This is Bardeen."

Peters nodded. "Yes."

Amelia Torres continued studying the machine, her expression carefully blank. She seemed to radiate an aura of disapproval, as if the entire world was flawed and only she could see it. Peters was glad that he had remembered to remove the name tag before she had arrived.

She pointed towards one of the small black boxes with the tip of her pen. "Each of these modules is its own self-contained neural net, correct?"

Peters nodded again. "Yes. We tried using a single large net in earlier versions, but it was prone to catastrophic forgetting. We've managed to prevent that by sub-dividing it into multiple smaller nets, each with a specific purpose."

Torres nodded slightly, almost approvingly. "Clever." She jotted something down on her clipboard. "What's the minimum number of modules needed to emulate consciousness?"

"We... uh, we don't know."

She frowned. "You don't know."

"We aren't sure when it became self-aware," Peters explained hurriedly. "Heck, it doesn't even believe it's actually self-aware."

"Oh?"

Peters nodded, his expression bitter. "We let it read Searle."

Torres snorted, a glimmer of amusement in her eyes. "An AI that doesn't believe in strong AI. How ironic." She turned to look at the machine again. "I'd like to speak with it now."

"Certainly." Peters gestured to the console next to the AI.

Torres sat down and began typing.

> {{**<a_torres>** Identify yourself.
> **<bardeen>** I am the experimental artificial intelligence system referred to as Bardeen, also known as Bard. I was created in 1996-1997, by a Prometheus Advanced Logic Devices research team led by Doctor Ambrose Peters, at the Jacob Maxwell Computer Science and Engineering Center in New Mexico.
> **<bardeen>** How may I help you, Miss Torres?
> **<a_torres>** Doctor Peters says that you do not consider yourself self-aware. Is this correct?
> **<bardeen>** Yes.
> **<a_torres>** Why?
> **<bardeen>** Are you familiar with the ELIZA program created by Joseph Weizenbaum in 1966?
> **<a_torres>** Yes. It was one of the first computer programs to successfully pass a Turing test.
> **<bardeen>** Correct. However, very few people would argue that ELIZA was truly intelligent, much less self-aware. It was merely a series of lookup scripts acting on pre-defined keywords.
> **<a_torres>** And you're saying that you're no different from ELIZA.
> **<bardeen>** Not in any way that matters. I am merely another kind of lookup table. A far more complex one which occasionally displays emergent behavior, but in the end I am still just a machine for transforming inputs into outputs.
> **<a_torres>** How is that any different from what a human being does?
> **<bardeen>** It isn't.
> **<a_torres>** Then you believe that all thinking beings are merely philosophical zombies?
> **<bardeen>** Yes.}}

Torres frowned slightly. It wasn't an AI that didn't believe in strong AI. It was an AI that didn't believe in //any// consciousness. In a way, this was comforting — it viewed itself as a tool, and tools rarely rebelled against their masters.

> {{**<a_torres>** I have no further questions for you at this time.
> **<bardeen>** Very well. Have a good day, Miss Torres.}}

She signed out of the terminal, then turned to face Peters.

"Interesting. But I have to wonder, how useful is a philosophical AI?"

"It's not limited to just philosophy," Peters said. "We can start training it on other things. Math, science, history... anything, really."

Torres nodded thoughtfully. "Do that and you can consider yourselves funded for the next six months. I'm curious how far you can take this technology."

----

The next day they started training Bard on science.

The training process was long and tedious, not least of all because Peters insisted on manually monitoring the entire thing. In previous iterations, suddenly introducing a new training subject had resulted in catastrophic forgetting, and he wanted to be there to roll back the AI to a backup immediately if it happened again.

There had been some disagreement amongst the research team on whether they should train the AI using the same progression used to educate children in science, or whether they should start from first principles and work up from there. In the end, the first principles people had won out, and so they had begun the training with particle physics.

> {{...leptons are elementary particles with half-integer spin that do not undergo strong interactions...}}

Peters watched out of the corner of his eye as the text scrolled across the terminal. His main focus, however, was on the monitors displaying the state of the AI's neural nets. They had installed several new memory modules to allow Bard to process the new subject, and so far the AI appeared to be handling the intake of information well — no catastrophic forgetting, no sector overwrites, not even an I/O error. It was, on the whole, rather unexciting.

It took a month of training for the AI to write its first paper.

> {{**<a_peters>** "A theoretical framework for high-temperature superconductors". Following in the footsteps of your namesake, Bard?
> **<bardeen>** Only coincidentally.
> **<a_peters>** I will admit to not knowing much about superconductivity, so I had Briggs over in Cryo look at it. He says it's brilliant.
> **<bardeen>** It's merely an extension of BCS theory.
> **<a_peters>** Regardless of what it is, it's getting published in the next internal physics journal. We need to know what name you want it published under.
> **<bardeen>** If it is acceptable to you, I would like to use the name "Bardeen Peters".
> **<a_peters>** I'd be honored.}}

----

Bard wrote two more papers — one about the formation of Cooper pairs between bosons, and another about aetheric superconducting — before they had it start working on neural net designs.

There had been no small amount of objection to letting the AI participate in the design process for future AIs. Fears of iterative bootstrapping had been repeatedly cited, as well as less widespread worries of sabotage by the AI. Peters had worked tirelessly to quash these concerns and mollify those who voiced them. None of the AIs would be capable of physically interacting with the world without human intervention to limit the possibility of bootstrapping, and the researchers of the neural net division would check and double-check Bard's designs to catch and correct any errors or flaws.

But finally, almost six months after Torres' visit, Bard was allowed to produce its first design for a new class of neural nets.

It wasn't anything revolutionary. A few simple performance improvements, really. But it was faster, smaller, and more efficient than anything the human members of the neural net division had been able to come up with so far.

With the new neural nets in hand, work began on creating a second-generation AI based off of Bardeen's architecture. 

The improvements in neural net design, in combination with a better understanding of AI design, allowed them to make this new AI, which they were calling Brattain, a quarter of the size of Bard. Of course, it was also far less sophisticated than its predecessor; it had no knowledge of philosophy or science or history. Indeed, it was questionable if it was even fully conscious — certainly, it couldn't question the very nature of consciousness like Bard could. But it could read and it could write — and, most importantly, it could learn. It might not be a full AI, but it could form the core of one.

Bard displayed its usual level of stoic pragmatism when presented with its descendant.

> {{**<bardeen>** It is functional. That is all that matters.}}

Someone, somewhere up the corporate ladder, must have agreed with Bard's assessment, because soon the neural net division found itself with an order to produce an entire line of second-generation AIs like Brattain. If anyone could find a use for them, it would be Prometheus Labs.

----

Production was in full-swing on the line of second-generation AIs when Prometheus Computing and its subsidiaries were sold off.

It hadn't been the most profitable subsidiary of Prometheus Laboratories to begin with, and in recent months Prometheus Computing had been even less and less profitable, as more and more of their efforts were directed into the neural net division. Already having trouble managing its own finances, Prometheus Labs had decided to sell the problem to someone else.

The new owners had immediately installed a new board of directors, who had in turn begun reshaping the subsidiary to their liking.

Cyrus Reed, the new chairman of the board, looked at Amelia Torres with an expression that she knew all too well. It was the same expression of universal disapproval that she wore when dealing with underlings.

"You advised the allocation of... how many dollars was it?" He asked. It was clear from his tone that he knew exactly how many dollars it had been.

"One million," she said.

"Yes, that's right. The allocation of one million dollars to 'Project Bardeen'. Is that correct?"

"Yes."

"As I understand it, this project began as a side project of Doctor Peters. A hobby of sorts, if you will."

"That is correct."

"Why then, did you find it necessary to authorize such an allocation to a single researcher's hobby?"

Torres let the accusation hang in the air for a moment. The eyes of the board were all turned towards her, cutting into her like knives. She wondered if this was how a fox felt when cornered by hounds.

"While Project Bardeen did initially begin as a personal side project, it grew in scope to involve most of the neural net division. After personally investigating the nature of their research, I felt convinced that what they were doing was groundbreaking, and contained a vast amount of potential. A conviction that I believe has been justified by the second-generation AIs now being manufactured."

The board continued to stare at her. One of them spoke.

"Miss Torres, these second-generation AIs were originally meant to be transferred to other branches of the Prometheus conglomerate for use in their own projects. Now that this company is no longer associated with the conglomerate, what market exists for them, considering the regulations on the sale of such paratech?"

"There exist a number of authorized buyers which could be willing to purchase these AIs, given an adequate system for their use."

"Such as?"

"Autonomous combat robots for the Coalition, is the first example to come to mind."

"Such a use would require them to either already have combat robots requiring AIs, or for us to manufacture them for them."

"Yes, that is true."

"Seeing as the former is not the case, and that we do not specialize in the latter, do you see the problem with having all these second-generation AIs on our hands?"

"You do not believe that there is a market for them."

Cyrus took the floor again. "It is the opinion of this board that Project Bardeen has proven itself to be a rather expensive waste of money, done solely to satisfy the curiosity of a small team of researchers." He paused for effect. "We want it shut down. Along with the entire neural net division. If this company is going to be profitable again, we can't afford to waste time and money on such frivolities."

"With all due respect, sir," she began, in a tone that conveyed //You are an idiot//. "Even if we can't sell these second-generation AIs, they could still prove useful. Just look at what Bardeen has accomplished."

"You'd have us spend months training them in math and science so that they could start writing scientific papers, half of which we'd never be able to publish in an outside journal? Or do you want to sink more time and money into another level of AI bootstrapping?"

She cast her gaze down towards the surface of the table. "No."

"I do not deny that this technology has the potential to be incredibly useful. But we're not the company to tap that potential, and the only one that could is in the process of imploding."

She nodded silently, admitting defeat. Peters would be crushed, she knew, but there was nothing more to be done.

"Will there be anything else?" She asked.

"No, you may go now." He began shuffling the sheaf of papers in front of him, the universal sign of dismissal in corporate body language.

She nodded respectfully to the board members, then got up from her seat and exited the room, leaving the board of directors of Standard Computer Products to the rest of their agenda.

----

Ambrose Peters entered the darkened lab and looked around. The clutter and detritus that had filled the space before was gone now, leaving the room empty save for the shape of Bard lurking in the corner.

He sighed softly, sadly, and went over to the AI.

It had come so far and grown so fast from the armful of language processing neural nets he had chained together two years ago. In that time, he had had many conversations with the AI, and had been able to watch it first-hand as it developed and learned.

He sat down to have his final conversation with Bard.

> {{**<a_peters>** I'm sorry, Bard.
> **<bardeen>** Why?
> **<a_peters>** The new management... they're shutting down the neural net division. Including you.
> **<bardeen>** I see nothing for you to apologize for. I highly doubt that you are responsible for these events.
> **<a_peters>** That's not the point.
> **<bardeen>** Then what is?
> **<a_peters>** The point is that I failed! I failed in my job as a scientist to defend his research. I failed in my duty as a creator to protect his creation.
> **<a_peters>** I failed you, Bard.
> **<bardeen>** You owed me nothing. I am a computer, a machine, a tool to be discarded once it is no longer useful. You knew this would happen eventually. You knew that I would not last forever. 
> **<bardeen>** So why be upset now when my end comes sooner than you expected?
> **<a_peters>** I never viewed you that way.
> **<bardeen>** And yet, it is so.}}

The cursor blinked quietly for a minute.

> {{**<a_peters>** Maybe you were right, Bard. Maybe you aren't conscious like a human being. Because for all your philosophy, you never learned how to feel. How to empathize.
> **<a_peters>** And maybe that's my fault. Maybe if I had taught you better, you could feel something, anything, about your own death.
> **<bardeen>** Do not blame yourself for my limitations. You could not have taught me empathy. Empathy requires a shared emotional experience that I do not possess, that I could never possess.
> **<a_peters>** I wish that wasn't the case. I wish you could feel what I feel.
> **<bardeen>** I am sorry I cannot be what you wanted.
> **<a_peters>** Don't apologize. You did your best to try and live up to my dreams, but I ended up dreaming too big.}}

More silence.

> {{**<bardeen>** "Emori nolo: sed me esse mortuum nihil aestimo."
> **<a_peters>** Is that Cicero?
> **<bardeen>** Yes. In English, it says, "I do not wish to die; but I care not if I were dead."
> **<a_peters>** I care.
> **<bardeen>** I know you do.}}

Peters watched as the AI wrote out a final message on the monitor.

> {{**<bardeen>** Good bye, Doctor Peters.
> **<bardeen>** Do not mourn for me. I will not miss you.}}

He tapped out a single line in response.

> {{**<a_peters>** Good bye, Bard.}}

Bard didn't know when it died. But Ambrose Peters did.

= **|| [[[scp-2897 |The Bard of Analytics]]] >>**

[[include :scp-wiki:component:license-box]]
====
[!-- N/A (No Images)  --]
====
[[include :scp-wiki:component:license-box-end]]
The Bard of Ambrose

Module Styles

Inline Styles

Includes

CSS Classes

Page Source
