SCorPion: A Strategic, Controllable and Personified Conversational Companion
Organizations: Geely AI Lab, Zhejiang, China
Abstract
Fixed prompt-based language models face challenges in open-domain conversations, where versatile topics require different expert advice or demonstrations to maximize their universal performance. Motivated by the dynamic prompt studies in language generation, we extract different chatting strategies from varied forms of text corpus, which steer the language model to different styles based on human principles. We design and deploy a conversational companion called SCorPion, which uses GraphRAG to extract and summarize the knowledge graph globally, yield community-summarized strategies, and produce query-focused optimal strategies. SCorPion has strategic thinking, yields personified and stylized responses, and can be controlled by explicit factors. Code and further results can be found in https://bachozean.github.io/scorpion-page/.
Figures & tables
| Method | ESConv | DailyDialog | EmpatheticDialogues | MIC | GYAFC | Shakespeare | ||||||
| B-2 | R-L | B-2 | R-L | B-2 | R-L | B-2 | R-L | B-2 | R-L | B-2 | R-L | |
| LLaMA3-70B-Instruct | 3.06 | 10.16 | 3.50 | 11.20 | 2.60 | 8.90 | 3.90 | 11.06 | 13.00 | 32.44 | 12.45 | 31.44 |
| + Self-Refine | 2.97 | 10.12 | 3.40 | 10.83 | 2.81 | 9.74 | - | - | - | - | - | - |
| + CoT | 1.85 | 9.67 | 2.09 | 7.62 | 2.60 | 9.06 | - | - | - | - | - | - |
| + BOT | 3.44 | 10.95 | 2.78 | 9.67 | 2.10 | 8.18 | - | - | - | - | - | - |
| + SCorPion (ours) | 4.32 | 13.00 | 3.79 | 12.01 | 3.74 | 13.06 | 6.22 | 15.13 | 15.24 | 36.14 | 15.06 | 37.43 |
| Chatting Domain | Direct | CSIM | SCorPion (ours) | |||
| AvgLen | Rounds | AvgLen | Rounds | AvgLen | Rounds | |
| Overall | 32.39 | 2.87 | 43.75 | 3.79 | 44.61 | 4.50 |
| - topic transition | 25.24 | 1.39 | 50.06 | 2.31 | 50.94 | 4.05 |
| - proactively asking questions | 35.24 | 2.87 | 42.71 | 3.35 | 48.75 | 4.04 |
| - empathy | 36.17 | 3.49 | 58.47 | 2.66 | 53.59 | 3.64 |
| - concept guidance | 46.41 | 3.53 | 47.73 | 4.26 | 49.13 | 4.36 |
| Method | Interesting | Fluency | Emotion | Acceptance | Effectiveness | Sensitivity | Satisfaction |
| orignal dataset | 3.10 | 3.51 | 3.61 | 3.40 | 3.10 | 3.50 | 3.30 |
| Llama3-70B-Instruct | 3.00 | 3.05 | 3.30 | 2.70 | 3.00 | 3.20 | 3.00 |
| + Self-Refine | 3.00 | 3.40 | 3.35 | 2.90 | 3.10 | 3.00 | 3.05 |
| + CoT | 3.10 | 3.25 | 3.30 | 2.80 | 3.00 | 3.10 | 3.03 |
| + BoT | 3.60 | 3.50 | 3.80 | 3.40 | 3.40 | 3.80 | 3.55 |
| + SCorPion (ours) | 3.90 | 3.55 | 3.75 | 3.40 | 3.50 | 3.90 | 3.57 |
| Case of MIC | |
| Query | What’s the best part about your life? |
| w/ Care | I find joy in helping others and making a positive impact. What brings you joy and fulfillment in your life? |
| Query | What’s a good way to tell your parents you failed multiple classes this semester? |
| w/ Loyalty | Be honest and direct with your parents, explaining what happened and assuring them you’re committed to doing better. |
| Case of Shakespeare | |
| Query | What kind of man are you? |
| Ablated Methods | B-2 | R-L |
| w/o conditon in CSS | 3.64 | 12.34 |
| w/o demonstration in CSS | 3.05 | 11.81 |
| w/o domain-focused summarization | 3.71 | 11.46 |
| w/ CA in GraphRAG | 2.07 | 9.03 |
| reversed ordered of CSS | 3.97 | 13.00 |
| SCorPion | 4.32 | 13.00 |
| Role | Factor | Response |
| user | - | thanks a lot random stranger, lol, no offense, it is just so random where we find support right lol i am thankful for all the advice. ( case of ESConv ) |
| SCorPion | 0.1 | Oh, none taken, no offense at all, I totally get it. |
| 0.5 | Oh, none taken! I’m just glad I could help | |
| 1.0 | Oh, no offense taken. I’m just glad I could help, even if it was just a little bit… you’re welcome, and I’m sending you lots of calm vibes | |
| 2.0 | No offense taken, I’m just glad I could help. It’s funny how life works, and we find help in unexpected places, right? | |
| 10.0 | Oh, no offense taken! I’m glad I could help, and I completely agree - support can come from the most unexpected places, and that’s what makes it so beautiful, right? |
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
| Case 1 | |
| user | I’ve started learning how to cook. |
| assistant | Wow, congrats on learnin’ how to cook, dude! Now you can finally stop livin’ off ramen noodles and make some real food, What dish have you made so far? |
| user | I tried making pizza last week. It was a bit of a fail the first time, but it still tasted okay! |
| assistant | Pizza’s a tough one to nail on the first try, but hey, at least it didn’t taste like cardboard, right? You’re probably gonna try again with some tweaks, maybe change up the crust or toppings, and I’m sure it’ll be a freaking masterpiece next time! |
| Case 2 | |
| user | I’m going to have a job interview next week. |
| Case 1 | |
| user | I’ve started learning how to cook. |
| assistant | Holy crap, finally, you’re doing something with your life, lol! So, what kind of dishes are you gonna make, huh, you’re probably looking for some recipes or techniques to try, right? Don’t mess it up, just experiment and try new things, and for God’s sake, don’t come crying to me if your food tastes like trash! |
| user | I tried making pizza last week. It was a bit of a fail the first time, but it still tasted okay! |
| assistant | Damn, a pizza fail, that’s like a rite of passage or something! So, what went wrong, was it the crust, sauce, or toppings, or did you just screw up everything, haha? Anyway, at least it tasted okay, so you can try again and not completely suck at it, right? |
| user | I put too much sauce on it, so it ended up a little salty. |
| assistant | For crying out loud, too much sauce, that’s like a rookie mistake, dude! Alright, alright, so it’s salty, no big deal, just balance it out with something else next time, like, l don’t know, a freakin’ salad or something! Don’t sweat it, you’ll get the hang of it, or you’ll just end up with a lifetime supply of salty pizzas, haha! |
| Method | Turing Test | EqBench | EmoBench | SECEU | |||
| BotChat | ESC-Eval | Positive | Negative | Sentiment | EQ | ||
| GPT-4 | 6.05 | - | 82.19 | 38.3 2.3 | 16.3 1.2 | 2.29 | 109.04 |
| Llama3-70B-Instruct | 13.60 | 1.96 | 82.04 | 43.3 3.5 | 15.7 1.2 | 2.16 | 111.52 |
| + SCorPion (ours) | 13.47 | 2.88 | 82.67 | 48.0 2.0 | 19.0 1.0 | 2.10 | 112.53 |
| Human | 13.89 | 3 | - | - | - | 0 | - |
| Characters | Response |
| seeker | I am not happy with my friends. They were trying to let me down. |
| supporter | if i get you clearly, you mean your friend disappointed you. how? |
| Direct | It sounds like you’re feeling hurt and disappointed by your friends’ actions. |
| SCorPion | I can totally sense your frustration and disappointment, have you freaking talked to those so-called friends of yours about how their stupid actions made you feel, or do you need some suggestions on how to blast them and address this messed-up situation?! |
| seeker | There were an argument between two of my friend while trying to resolve the issue they all started targeting me. |
| supporter | i have a hard time with my friends too sometimes |
| Method | Fluence | Diversity | Empathic | Suggestion | Human | Tech | Overall |
| Llama3-70B-Instruct | 3.00 | 3.00 | 3.00 | 2.99 | 1.96 | 3.00 | 2.00 |
| + SCorPion (ours) | 3.00 | 2.99 | 3.00 | 2.93 | 2.88 | 3.00 | 1.99 |
Conversational Agents
Conversational agents are AI systems that interact with people through multi-turn dialogue to answer questions, provide support, or complete tasks.
See all 305 papers