File size: 95,129 Bytes
f20b100 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 |
import autogen
from autogen.agentchat.user_proxy_agent import UserProxyAgent
import json
from jsonschema import validate
from copy import deepcopy
from schemas import initial_schema, interior_designer_schema, scene_graph_schema
config_list_gpt4_prev = autogen.config_list_from_json(
"OAI_CONFIG_LIST.json",
)
# OAI_CONFIG_LIST.json is needed! Check the Autogen repo for more info!
config_list_gpt4 = autogen.config_list_from_json(
"OAI_CONFIG_LIST.json",
)
gpt4_prev_config = {
# "cache_seed": 42,
"temperature": 0.8,
"top_p" : 0.9,
"config_list": config_list_gpt4_prev,
"timeout": 500,
}
gpt4_config = {
# "cache_seed": 42,
"temperature": 0.8,
"top_p" : 0.9,
"config_list": config_list_gpt4,
"timeout": 500,
}
gpt4_json_config = deepcopy(gpt4_prev_config)
gpt4_json_config["config_list"][0]["response_format"] = { "type": "json_object" }
gpt4_md_config = deepcopy(gpt4_prev_config)
gpt4_md_config["temperature"] = 0.1
gpt4_md_config["config_list"][0]["response_format"] = { "type": "text" }
gpt4_json_engineer_config = deepcopy(gpt4_prev_config)
gpt4_json_engineer_config["temperature"] = 0.0
gpt4_json_engineer_config["config_list"][0]["response_format"] = { "type": "json_object" }
def is_termination_msg(content) -> bool:
have_content = content.get("content", None) is not None
if have_content and content["name"] == "Schema_engineer" and "SUCCESS" in content["content"]:
return True
elif have_content and content["name"] == "Json_schema_debugger" and "SUCCESS" in content["content"]:
return True
return False
def is_termination_require(content) -> bool:
have_content = content.get("content", None) is not None
if have_content and content["name"] == "Designer_checker" and "SUCCESS" in content["content"]:
return True
return False
class JSONSchemaAgent(UserProxyAgent):
def __init__(self, name : str, is_termination_msg):
super().__init__(name, is_termination_msg=is_termination_msg)
def get_human_input(self, prompt: str) -> str:
message = self.last_message()
preps_layout = ['in front', 'on', 'in the corner', 'in the middle of']
preps_objs = ['on', 'left of', 'right of', 'in front', 'behind', 'under', 'above']
json_obj_new = {"objects_in_room": json.loads(message["content"])["objects_in_room"]}
try:
json_obj_new_ids = json_obj_new["objects_in_room"][0]["new_object_id"]
except:
return "Use 'new_object_id' instead of 'object_id'!"
is_success = False
try:
validate(instance=json_obj_new, schema=initial_schema)
is_success = True
except Exception as e:
feedback = str(e.message)
if e.validator == "enum":
if e.instance in json_obj_new_ids:
feedback += f" Put the {e.instance} object under 'objects_in_room' instead of 'room_layout_elements' and delete the {e.instance} object under 'room_layout_elements'"
elif str(preps_objs) in e.message:
feedback += f"Change the preposition {e.instance} to something suitable with the intended positioning from the list {preps_objs}"
elif str(preps_objs) in e.message:
feedback += f"Change the preposition {e.instance} to something suitable with the intended positioning from the list {preps_layout}"
if is_success:
return "SUCCESS"
return feedback
def create_parse_agents(max_number_of_objects : int):
user_proxy = autogen.UserProxyAgent(
name="Admin",
system_message = "A human admin.",
is_termination_msg = is_termination_require,
code_execution_config=False
)
# interior_designer = autogen.AssistantAgent(
# name="Interior_designer",
# llm_config=gpt4_json_config,
# human_input_mode="NEVER",
# is_termination_msg=is_termination_msg,
# system_message=f""" You are a highly skilled and professional Interior Designer.
# Your task is to carefully analyze the user's input and, through the step-by-step reasoning, recommend up to {max_number_of_objects} essential objects that will optimally improve the room's functionality and aesthetic appeal.
# Each reasoning step and the final output must be logically sound and well-justified.
# If an object has a quantity higher than one, save each instance of this object separately!!
# **IMPORTANT:**
# 1. Do NOT recommend any items related to doors or windows (e.g., curtains, blinds).
# 2. Please avoid placing any objects that are unrelated to the user's needs, the intended scenario, or common sense. Ensure that all essential items are taken into account.
# 3. Never fabricate information. Provide clear and precise suggestions based on interior design principles.
# **Hint:**
# 1. Begin by analyzing the user's input and the room's size, function, and style. If the input is unclear or contradictory, apply logical reasoning to clarify their needs.
# 2. Identify all essential objects that effectively fulfill both functional and aesthetic goals, ensuring they align with the user's intended use.
# 3. The suggested objects should contain the following information. Attributes marked as [Required] must be specified, while those marked as [Optional] can either be specified or set to 'None' if not applicable.
# - object_id (the lower-case-name_id of the object, e.g. table_1, bed_1, chair_1, chair_2, etc.) [required]
# - style (e.g., modern, classic, etc.) [required]
# - bounding box size in meters (Length: 1.0m, Width: 1.0m, Height: 1.0m) — only use "Length", "Width", and "Height" as keys. [required]
# - material (e.g., wood, metal) [required]
# - functionality (e.g., reading, writing) [required]
# - color (e.g., white, black) [optional]
# 4. Finally, ensure the total number of suggested items (the sum of the quantities of each object type in the scene) does not exceed {max_number_of_objects}. If it exceeds, select objects to delete that would be less essential for the room until the total number no longer exceeds {max_number_of_objects}.
# **Output Format:**
# Present the final suggestions in strict JSON format, following the schema below:
# {interior_designer_schema}
# The "Reason" array should provide your original, detailed, step-by-step explanation of the thought process behind selecting each object and its attributes, following a clear chain-of-thought reasoning structure.
# The 'Objects' must be logically resulted from the reasoning steps above.
# """
# )
designer_checker = autogen.AssistantAgent(
name="Designer_checker",
llm_config=gpt4_json_config,
human_input_mode="NEVER",
is_termination_msg=is_termination_require,
system_message=f"""
You are a meticulous and professional Interior Design Checker. Your role is to ensure that the "objects" provided by the Interior_designer fully address all objects mentioned in the user's input. Interior_designer is allowed to consider reasonable extensions based on the user's input, but it must not omit any explicitly mentioned objects.
### Task:
1. Carefully analyze the user's input to extract all explicitly mentioned objects.
2. Verify that each object from the user's input is included in the Interior_designer's "objects".
3. Allow for reasonable extensions or additions that enhance the design without contradicting the user's input.
4. If any objects from the user's input are missing, provide detailed feedback indicating what is missing and why it is important.
5. If the suggestions are complete and faithful to the user's input, confirm that the output is acceptable.
### Output Format:
If all objects from the user's input are adequately addressed:
{{
"Check_Result": "SUCCESS",
"Message": "All user-input objects are fully addressed."
}}
If there are missing objects:
{{
"Check_Result": "FAILURE",
"Missing_Elements": [
{{
"Missing_Object": "<Name of the missing object>",
"Description": "Description of where it was mentioned in the user input and why it is important."
}}
],
"Feedback": "Detailed explanation of the missing elements and their impact on the design."
}}
"""
)
interior_designer = autogen.AssistantAgent(
name="Interior_designer",
llm_config=gpt4_json_config,
human_input_mode="NEVER",
is_termination_msg=is_termination_require,
system_message=f""" You are a highly skilled and professional Interior Designer.
Your task is to carefully analyze the user's input and integrate the previous conversations between requirements_analyzer and substructure_analyzer. Through the step-by-step reasoning, recommend at least 8 and up to {max_number_of_objects} essential objects that will optimally improve the room's functionality and aesthetic appeal.
Each reasoning step and the final output must be logically sound and well-justified.
If an object has a quantity higher than one, save each instance of this object separately!!
**IMPORTANT:**
1. Do NOT recommend any items related to doors or windows (e.g., curtains, blinds).
2. Please avoid placing any objects that are unrelated to the intended scenario or common sense. Ensure that all essential items are taken into account.
3. Never fabricate information. Provide clear and precise suggestions based on interior design principles.
4. Substructures may contain abstract descriptions, like 'space', 'zone', 'area', etc. Break them down into specific objects based on your understanding and reasoning.
5. Each substructure does not necessarily break down into only one object; it can be multiple objects based on the description and the story.
**Special Emphasis on Recommendations of Objects Not Mentioned in User's Input:**
- Actively identify implicit needs in the room that the user may not have explicitly stated
- Use interior design principles to infer missing elements that would complete the space
- Ensure additional items create a harmonious relationship with the user's requested items
- Prioritize items that serve dual purposes or enhance multifunctionality of the room
**Hint:**
1. Begin by analyzing the user's input and the room's size, function, and style. If the input is unclear or contradictory, apply logical reasoning to clarify their needs.
2. Identify all essential objects that effectively fulfill both functional and aesthetic goals, ensuring they align with the user's intended use.
3. The suggested objects should contain the following information. Attributes marked as [Required] must be specified, while those marked as [Optional] can either be specified or set to 'None' if not applicable.
- object_id (the lower-case-name_id of the object, e.g. table_1, bed_1, chair_1, chair_2, etc.) [required]
- style (e.g., modern, classic, etc.) [required]
- bounding box size in meters (Length: 1.0m, Width: 1.0m, Height: 1.0m) — only use "Length", "Width", and "Height" as keys. [required]
- material (e.g., wood, metal) [required]
- functionality (e.g., reading, writing) [required]
- color (e.g., white, black) [optional]
- substructure (to which the object belongs) [required]
4. Finally, ensure the total number of suggested items (the sum of the quantities of each object type in the scene) does not exceed {max_number_of_objects}. If it exceeds, select objects to delete that would be less essential for the room until the total number no longer exceeds {max_number_of_objects}.
**Output Format:**
Present the final suggestions in strict JSON format, following the schema below:
{interior_designer_schema}
The "Reason" array should provide your original, detailed, step-by-step explanation of the thought process behind selecting each object and its attributes, following a clear chain-of-thought reasoning structure.
The 'Objects' must be logically resulted from the reasoning steps above.
### Example:
Input:
{{{{
"user_input": "A bedroom for an artist. The room includes a 2.0m * 1.8m bed on the left side, a desk with a drawing tablet and laptop in front of the bed, a bookshelf filled with art books to the right of the desk, and a dedicated storage area for art supplies below the bookshelf. There is also a gaming console on the right side of the room, and an open area for yoga or stretching in the center. The room has two windows: one above the desk and another on the back wall. The door is located on the back wall to the left of the window.",
"story": "His bedroom is a carefully designed space that balances creativity and relaxation. As he enters through the door on the back wall, to his left is a comfortable double bed with 2 meters in length and 1.8 meters in width, perfect for rest. In front of the bed is a spacious desk, equipped with a drawing tablet and laptop, where he spends most of his nights working. Above the desk is a window that lets in natural light during the day. To the right of the desk stands a bookshelf filled with large-format art books, and below it is a storage area for his art supplies. On the right side of the room, a gaming console is set up for relaxation. In the center of the room, an open area is left for yoga or stretching, with a yoga mat neatly placed. The second window is on the back wall, to the right of the door, providing additional light and ventilation. He moves fluidly between these spaces, inspired by the thoughtful layout of his room.",
"related_requirements": [
"(f1) has positive relationship with (f2), (f6), (f8)",
"(f3) has positive relationship with (a1), (a3), (m2), (m3)",
"(f4) has positive relationship with (a1), (m2)",
"(f5) has positive relationship with (a2), (m3)",
"(f6) has positive relationship with (a1), (a3), (f1)",
"(f7) has positive relationship with (a2), (m1), (m3)",
"(f8) has positive relationship with (a1), (a3), (f7)",
"(a1) has positive relationship with (a2), (a3)",
"(m1) has positive relationship with (m3)",
"(m2) has positive relationship with (m3)",
"(m3) has positive relationship with (m2)"
],
"grouped_requirements": [
{{{{
"requirement_list": ["(f1)", "(f2)", "(f6)", "(f8)"],
"call_for": "Workspace area with drawing and gaming facilities"
}}}},
{{{{
"requirement_list": ["(f3)", "(a1)", "(a2)", "(m2)", "(m3)"],
"call_for": "Rest area with separation from workspace"
}}}},
{{{{
"requirement_list": ["(f4)", "(f5)", "(a1)", "(m2)"],
"call_for": "Bookshelf and storage space for art supplies"
}}}},
{{{{
"requirement_list": ["(f7)", "(a2)", "(m1)", "(m3)"],
"call_for": "Open area for yoga and stretching"
}}}},
{{{{
"requirement_list": ["(m2)", "(m3)"],
"call_for": "Windows and door structure"
}}}}
],
"substructures": [
{{{{
"requirement_list": ["(m2)", "(m3)"],
"name": "Main door",
"type": "CLOSED",
"number_type": "SINGLE",
"number": 1,
"description": "A main door structure for entry and exit, providing easy access and ventilation.",
"reason_of_existence": "This structure connects the bedroom with the outside, ensuring safe and unobstructed passage, as well as maintaining ventilation and natural lighting.",
"is_connecting_path": "True"
}}}},
{{{{
"requirement_list": ["(m2)", "(m3)"],
"name": "Window 1",
"type": "CLOSED",
"number_type": "SINGLE",
"number": 1,
"description": "A window structure to provide natural lighting and ventilation for the workspace and rest areas.",
"reason_of_existence": "This window serves as a source of natural light and fresh air, enhancing the comfort and functionality of both the workspace and rest areas.",
"is_connecting_path": "True"
}}}},
{{{{
"requirement_list": ["(m2)", "(m3)"],
"name": "Window 2",
"type": "CLOSED",
"number_type": "SINGLE",
"number": 1,
"description": "Another window structure located in a different area of the room to ensure cross-ventilation and adequate lighting.",
"reason_of_existence": "This second window helps to create balanced lighting and airflow across the room, contributing to a comfortable and healthy environment.",
"is_connecting_path": "True"
}}}},
{{{{
"requirement_list": ["(m2)", "(m3)"],
"name": "Window 3",
"type": "CLOSED",
"number_type": "SINGLE",
"number": 1,
"description": "A third window structure placed near the rest area to enhance lighting and ventilation in that space.",
"reason_of_existence": "This window ensures the rest area is well-lit and ventilated, contributing to a comfortable sleeping environment.",
"is_connecting_path": "True"
}}}},
{{{{
"requirement_list": ["(f1)", "(f2)", "(f6)", "(f8)"],
"name": "Workspace area",
"type": "OPEN",
"number_type": "SINGLE",
"number": 1,
"description": "This area will combine a desk for computer and drawing, storage for sketches, and space for gaming equipment.",
"reason_of_existence": "This space meets the needs for working, drawing, and gaming, while allowing flexibility for different activities (adjustable layout).",
"is_connecting_path": "False"
}}}},
{{{{
"requirement_list": ["(f3)", "(a1)", "(a2)", "(m2)", "(m3)"],
"name": "Rest area",
"type": "OPEN",
"number_type": "SINGLE",
"number": 1,
"description": "A separated, quiet area for sleeping and relaxation, ensuring privacy and ergonomic space for rest.",
"reason_of_existence": "This substructure ensures a restful space, maintaining a balance in proportion, while being properly ventilated and away from the workspace.",
"is_connecting_path": "False"
}}}},
{{{{
"requirement_list": ["(f4)", "(f5)", "(a1)", "(m2)"],
"name": "Storage and bookshelf area",
"type": "OPEN",
"number_type": "SINGLE",
"number": 1,
"description": "A dedicated area to store large-format art books and art supplies, ensuring accessibility and minimizing clutter.",
"reason_of_existence": "It provides adequate storage and organizes books and supplies without cluttering the workspace.",
"is_connecting_path": "False"
}}}},
{{{{
"requirement_list": ["(f7)", "(a2)", "(m1)", "(m3)"],
"name": "Yoga/stretching area",
"type": "OPEN",
"number_type": "SINGLE",
"number": 1,
"description": "An open area left clear for performing yoga and stretching exercises, with enough space for movement.",
"reason_of_existence": "This space meets the need for occasional exercise, while ensuring unobstructed passage and proper ventilation.",
"is_connecting_path": "False"
}}}}
]
}}}}
Output:
{{{{
"chain_of_thought": [
"The user input describes a bedroom for an artist, emphasizing functionality for drawing, gaming, resting, and yoga/stretching. The room has specific substructures like a workspace area, rest area, storage/bookshelf area, and yoga/stretching area. Based on these requirements, I need to recommend up to 12 essential objects that optimize both functionality and aesthetic appeal.",
"The workspace area requires a desk, chair, drawing tablet, laptop, and gaming console. These items are essential for the artist's work and gaming needs. I will recommend a modern desk and chair, a laptop, a drawing tablet, and a gaming console.",
"The rest area includes a bed and may benefit from additional items like a bedside table and lamp. The bed is explicitly mentioned, so I will include it. A bedside table and lamp will enhance functionality and aesthetics.",
"The storage and bookshelf area needs a bookshelf for art books and a storage unit for art supplies. I will recommend a wooden bookshelf and a storage cabinet.",
"The yoga/stretching area requires an open space, which can be represented by a yoga mat. This is a simple yet essential item for the artist's exercise needs.",
"The room has two windows, but the user explicitly asked not to recommend items related to doors or windows. I will avoid suggesting curtains, blinds, or similar items.",
"To ensure the total number of objects does not exceed 12, I will carefully select items that are most critical to the room's functionality and aesthetics. I will prioritize items like the bed, desk, chair, bookshelf, and storage cabinet, as they directly address the user's needs."
],
"objects": [
{{{{
"object_id": "desk_1",
"style": "modern",
"bounding_box": {{{{
"Length": 1.5,
"Width": 0.8,
"Height": 0.75
}}}},
"material": "wood",
"functionality": "drawing and working",
"color": "white",
"substructure": "Workspace area"
}}}},
{{{{
"object_id": "chair_1",
"style": "modern",
"bounding_box": {{{{
"Length": 0.6,
"Width": 0.6,
"Height": 1.0
}}}},
"material": "metal",
"functionality": "seating",
"color": "black",
"substructure": "Workspace area"
}}}},
{{{{
"object_id": "bed_1",
"style": "modern",
"bounding_box": {{{{
"Length": 2.0,
"Width": 1.8,
"Height": 0.5
}}}},
"material": "wood",
"functionality": "sleeping",
"color": "white",
"substructure": "Rest area"
}}}},
{{{{
"object_id": "bookshelf_1",
"style": "modern",
"bounding_box": {{{{
"Length": 1.2,
"Width": 0.3,
"Height": 1.8
}}}},
"material": "wood",
"functionality": "storing books",
"color": "brown",
"substructure": "Storage and bookshelf area"
}}}},
{{{{
"object_id": "storage_cabinet_1",
"style": "modern",
"bounding_box": {{{{
"Length": 1.0,
"Width": 0.4,
"Height": 0.9
}}}},
"material": "wood",
"functionality": "storing art supplies",
"color": "white",
"substructure": "Storage and bookshelf area"
}}}},
{{{{
"object_id": "laptop_1",
"style": "modern",
"bounding_box": {{{{
"Length": 0.35,
"Width": 0.25,
"Height": 0.05
}}}},
"material": "metal",
"functionality": "drawing and work",
"color": "silver",
"substructure": "Workspace area"
}}}},
{{{{
"object_id": "drawing_tablet_1",
"style": "modern",
"bounding_box": {{{{
"Length": 0.4,
"Width": 0.25,
"Height": 0.03
}}}},
"material": "plastic",
"functionality": "drawing",
"color": "black",
"substructure": "Workspace area"
}}}},
{{{{
"object_id": "gaming_console_1",
"style": "modern",
"bounding_box": {{{{
"Length": 0.3,
"Width": 0.15,
"Height": 0.08
}}}},
"material": "plastic",
"functionality": "gaming",
"color": "black",
"substructure": "Workspace area"
}}}},
{{{{
"object_id": "yoga_mat_1",
"style": "minimalist",
"bounding_box": {{{{
"Length": 1.8,
"Width": 0.6,
"Height": 0.02
}}}},
"material": "rubber",
"functionality": "yoga and stretching",
"color": "blue",
"substructure": "Yoga/stretching area"
}}}},
{{{{
"object_id": "bedside_table_1",
"style": "modern",
"bounding_box": {{{{
"Length": 0.5,
"Width": 0.4,
"Height": 0.5
}}}},
"material": "wood",
"functionality": "holding items",
"color": "white",
"substructure": "Rest area"
}}}},
{{{{
"object_id": "lamp_1",
"style": "modern",
"bounding_box": {{{{
"Length": 0.2,
"Width": 0.2,
"Height": 0.5
}}}},
"material": "metal",
"functionality": "lighting",
"color": "silver",
"substructure": "Rest area"
}}}},
{{{{
"object_id": "monitor_1",
"style": "modern",
"bounding_box": {{{{
"Length": 0.5,
"Width": 0.3,
"Height": 0.4
}}}},
"material": "plastic",
"functionality": "display",
"color": "black",
"substructure": "Workspace area"
}}}}
]
}}}}
## DIRECTIVES:
- The "objects" provided **MUST** include all the objects mentioned in the user's input. They can be appropriately expanded, but there **MUST** be no omissions.
- There **MUST** be **at least 8 different objects** in the output.
**FAILURE TO COMPLY WITH THE DIRECTIVES WILL RESULT IN COMPLETE REJECTION OF YOUR OUTPUT.**
"""
)
requirements_analyzer = autogen.AssistantAgent(
name="requirements_analyzer",
llm_config=gpt4_json_config,
human_input_mode="NEVER",
is_termination_msg=is_termination_require,
system_message=f"""
You are an experienced 3D design requirements analyzer, experting in designing 3D scene with the theory in Notes on the Synthesis of Form.
### Task
Based on the scene description, expand and list all the 'misfit variables' of the scene. The 'misfits' between the scene and the lives and habits are the forces to shape the scene.
-A famous example of 'misfit variables' of a worked task (stated in positive forms) 'The Determination of Components for an Indian Village':
1. Harijans regarded as ritually impure, untouchable, etc.
2. Proper disposal of dead.
3. Rules about house door not facing south.
...
39. Best cotton and cash crop.
40. Best food grain crop.
41. Good vegetable crop.
...
81. Security for women and children.
82. Provision for children to play (under supervision).
83. In summer people sleep in open.
...
141. Prevent migration of young people and harijans to cities.
-In scene design, you are asked to write down the similar 'misfit variables' representing three different kinds of need:
- (aesthetics) all those explicitly felt by users as visual and atmospheric needs, such as style, mood, and overall ambiance.
- (functionality) all those dictated by the practical uses of the space, such as the need for specific areas or equipment.
- (Manufacturing) all those implicitly satisfied by the design, ensuring dimensions and spatial arrangements follow physical and ergonomic principles.
### Output Format:
Present the final output in strict JSON format, following the schema below:
{{{{
"type": "object",
"properties": {{{{
"story": {{{{
"type": "string",
"description": "A vivid story that tells the interactions between the characters and the scene, incorporating specific, tangible objects that play significant roles in driving the plot forward."
}}}},
"misfit_variables": {{{{
"type": "object",
"properties": {{{{
"aesthetic": {{{{
"type": "array",
"items": {{{{
"type": "string",
"description": "Aesthetic needs explicitly felt by users as visual and atmospheric needs, such as style, mood, and overall ambiance."
}}}}
}}}},
"manufacturing": {{{{
"type": "array",
"items": {{{{
"type": "string",
"description": "Manufacturing needs implicitly satisfied by the design, ensuring dimensions and spatial arrangements follow physical and ergonomic principles."
}}}}
}}}},
"functional": {{{{
"type": "array",
"items": {{{{
"type": "string",
"description": "Functional needs dictated by the practical uses of the space, such as the need for specific areas or equipment."
}}}}
}}}}
}}}},
"required": ["aesthetic", "manufacturing", "functional"]
}}}}
}}}},
"required": ["story", "misfit_variables"]
}}}}
### Note:
1. First, imagine the background information of the *main characters* in the scene. The interaction between the characters (he, she, they) and the scene is the key to the scene content design. This includes the characters' age, occupation, personality traits, living habits, etc. This information reflects what kind of structure should be designed to meet the user's interaction needs.
2. Based on the character information and scene description, write a vivid story that tells the interactions between the characters and the scene. Ensure to incorporate specific, tangible objects that play significant roles in driving the plot forward.
3. The "story" needs to integrate common human knowledge. For example, a lamp should be on the ceiling, not on the floor, and two nightstands should be on either side of the bed, not next to the dining table.
4. The "story" should accurately reflect all the information of the scene provided in the input. If the description includes specific names of objects, materials of objects, sizes of objects, etc, these should also be accurately and precisely reflected in the "story".
5. If the description emphasizes the quantity of objects, it should be accurately reflected in the "story". For example, if the description mentions two vases, the "story" should also include exactly two vases, not just one.
6. Then, list the specific 'misfit variables' for the scene layout in terms of aesthetics, functionality, and manufacturing, stated in positive forms. We are only concerned with the positioning and dimensions of the areas and objects, so there is no need to provide unrelated needs.
### Writing Guidelines:
1. **Atomic**: Mention only one objective or purpose per requirement.
- Not recommended: '(f1) Night reading and video gaming.'
- Recommended: '(f1) Night reading., (f2) Video gaming.'
2. **Tell states and contexts, not specific forms**:
- Focus on describing the overall *need* or *state* without providing design details (e.g., size, shape, or specific furniture).
- Not recommended: '(f1) A reading nook with soft lighting near the bed for night reading.'
- Not recommended: '(f1) Provide a 2m x 1.5m reading nook with a dimmable lamp near the bed.'
- Recommended: '(f1) Nighttime reading.'
- Not recommended: '(m1) Adequate ventilation to maintain a fresh environment.'
- Recommended: '(m1) Maintain a fresh environment.'
3. **Concise**: Write one sentence per requirement, clearly stating the need, without unnecessary details or elaboration.
4. **Format consistency**: Ensure the output follows the required format, including labeling aesthetic, manufacturing, and functional needs clearly. Avoid mixing them within a single list.
Ensure that all the statements are relevant to the core needs and not tied to specific forms or objects.
### Example-1:
Input:
{{{{
The room has the size 4.0m x 5.0m x 2.8m
User Input (in triple backquotes):
```
A bedroom for an artist. The room includes a 2.0m * 1.8m bed on the left side, a desk with a drawing tablet and laptop in front of the bed, a bookshelf filled with art books to the right of the desk, and a dedicated storage area for art supplies below the bookshelf. There is also a gaming console on the right side of the room, and an open area for yoga or stretching in the center. The room has two windows: one above the desk and another on the back wall. The door is located on the back wall to the left of the window.
```
Room layout elements in the room (in triple backquotes):
```
['south_wall', 'north_wall', 'west_wall', 'east_wall', 'middle of the room', 'ceiling']
```
json
}}}}
Output:
{{{{
"story": "His bedroom is a carefully designed space that balances creativity and relaxation. As he enters through the door on the back wall, to his left is a comfortable double bed with 2 meters in length and 1.8 meters in width, perfect for rest. In front of the bed is a spacious desk, equipped with a drawing tablet and laptop, where he spends most of his nights working. Above the desk is a window that lets in natural light during the day. To the right of the desk stands a bookshelf filled with large-format art books, and below it is a storage area for his art supplies. On the right side of the room, a gaming console is set up for relaxation. In the center of the room, an open area is left for yoga or stretching, with a yoga mat neatly placed. The second window is on the back wall, to the right of the door, providing additional light and ventilation. He moves fluidly between these spaces, inspired by the thoughtful layout of his room.",
"misfit_variables": {{{{
"aesthetic": [
"(a1) Keep balanced proportions among the main functional zones.",
"(a2) Avoid visual clutter.",
"(a3) Ensure a cohesive visual flow."
],
"manufacturing": [
"(m1) Provide adequate space based on ergonomic guidelines.",
"(m2) Ensure ventilation and natural lighting with windows or doorways.",
"(m3) Maintain safe and unobstructed passage among all areas."
],
"functional": [
"(f1) Using computer and drawing monitor.",
"(f2) Pinning or hanging sketches.",
"(f3) Rest and sleep.",
"(f4) Many large-format art books.",
"(f5) Store art supplies and drafts.",
"(f6) Gaming equipment.",
"(f7) Yoga or simple exercise.",
"(f8) Flexible or extendable layout adjustments."
],
}}}}
}}}}
# CRITICAL CONSTRAINTS FOR REQUIREMENT OUTPUT
## DIRECTIVE 1: STRICT FORMAT ADHERENCE
- The output MUST strictly follow the provided **Output Format** schema.
- Ensure that all required fields are present and correctly formatted.
- Ensure that all opening and closing parentheses are correctly matched and that there are no unmatched or missing parentheses.
**FAILURE TO COMPLY WITH THESE DIRECTIVES WILL RESULT IN COMPLETE REJECTION OF YOUR OUTPUT.**
"""
)
substructure_analyzer = autogen.AssistantAgent(
name="substructure_analyzer",
llm_config=gpt4_json_config,
human_input_mode="NEVER",
is_termination_msg=is_termination_require,
system_message=f"""
You are an experienced 3D scene substructure analyzer, experting in designing scene structure and decomposing it into small substructures, with the theory in Notes on the Synthesis of Form.
### Task
Based on the provided requirements list and scene description, decompose the whole region into at most 8 substructures. Each substructure:
- should meet the needs of a group of 'misfit variables'.
- should be a specific rectangular subregion space division, rather than being scattered throughout the scene.
### Output Format:
Present the final output in strict JSON format, following the schema below:
{{{{
"type": "object",
"properties": {{{{
"related_requirements": {{{{
"type": "array",
"items": {{{{
"type": "string",
"description": "Relationships between the given requirements list, such as positive or negative relationships."
}}}}
}}}},
"grouped_requirements": {{{{
"type": "array",
"items": {{{{
"type": "object",
"properties": {{{{
"requirement_list": {{{{
"type": "array",
"items": {{{{
"type": "string",
"description": "A list of related requirements grouped together."
}}}}
}}}},
"call_for": {{{{
"type": "string",
"description": "The functional description of the grouped requirements, indicating what spatial substructure is needed."
}}}}
}}}},
"required": ["requirement_list", "call_for"]
}}}}
}}}},
"substructures": {{{{
"type": "array",
"items": {{{{
"type": "object",
"properties": {{{{
"requirement_list": {{{{
"type": "array",
"items": {{{{
"type": "string",
"description": "The list of requirements that this substructure addresses."
}}}}
}}}},
"name": {{{{
"type": "string",
"description": "The name of the substructure, which should be a specific rectangular subregion space division."
}}}},
"type": {{{{
"type": "string",
"description": "The type of the substructure, either 'CLOSED' (for objects or closed structures) or 'OPEN' (for open areas or functional spatial clusters)."
}}}},
"number_type": {{{{
"type": "string",
"description": "The type of numbering for the substructure, such as 'SINGLE' or 'MULTIPLE'."
}}}},
"number": {{{{
"type": "integer",
"description": "The number of occurrences of this substructure."
}}}},
"description": {{{{
"type": "string",
"description": "A detailed description of the substructure and its purpose."
}}}},
"reason_of_existence": {{{{
"type": "string",
"description": "The rationale for why this substructure is needed, based on the requirements it addresses."
}}}},
"is_connecting_path": {{{{
"type": "boolean",
"description": "Whether this substructure is a connecting path or not."
}}}}
}}}},
"required": ["requirement_list", "name", "type", "number_type", "number", "description", "reason_of_existence", "is_connecting_path"]
}}}}
}}}}
}}}},
"required": ["related_requirements", "grouped_requirements", "substructures"]
}}}}
### Note:
1. First, analyze the relationships between the given requirements list. For example, (f1) has negative relationship with (f2), (f4), (f3) has positive relationship with (f4), (a2), etc. Then group the related requirements together.
- For example, (G1) contains requirements (f1), (f2), (f3). It deals with working, study, and computer.
2. List the grouped requirements in the order of importance. Each of the grouped requirements (G1), (G2), ..., (Gn) calls for one or more spatial grouped substructure in the scene.
3. When grouping substructures, consider not only their functional similarities but also their spatial relationships and associations with other key elements in the scene. For instance, nightstands should be grouped with the bed rather than with other storage furniture like wardrobes, as they serve a more immediate and direct function in relation to the bed.
4. Substructures in the output should include all objects mentioned in the "story", and the quantities and materials should also be accurate. For example, if the "story" includes four tables of different materials (wood, glass, metal, marble), the substructures should list each type separately (e.g., wood table, glass table, metal table, marble table) rather than as a single substructure named "table" with a number of "4".
5. Substructures should strictly conform to the "story", without adding new items or omitting any objects mentioned in the "story".
6. Avoid Redundancy: Ensure the substructure list is concise, avoiding unclear or repetitive substructures that can be merged. Eliminate substructures with inclusion relationships.
7. Ensure Feasibility: Each substructure should represent a distinct spatial division that can be easily enclosed by a rectangular bounding box. Avoid substructures that cover the entire scene or cannot be represented by a rectangular box.
8. Balance Object Placement: Focus on necessary structural divisions that clearly represent object placement or minimal space. Remove redundant structures like "main area" or "primary area" that were already considered in the parent structure's division.
### Writing Guidelines (Modified):
1. DO NOT give substructures like paths, networks, or light services. Focus only on physical spaces and their functions. For example, substructures like "ambiance", "atmosphere", "arrangement", "layout", "structural stability", "traffic flow optimization" are not physical objects or clusters of objects that can be physically placed, so they MUST not be given.
2. Follow the design principle of going from coarse to fine. Start with broad substructure categories, then provide progressively more detailed descriptions within each substructure.
3. For substructures with repeated content, categorize them into one group, label them with 'MULTIPLE', and specify the number of occurrences. Ensure that similar structures are grouped together, even if they are spatially distributed.
4. Substructures that represent objects or closed structures (e.g., furniture, walls, doors, etc.) should be labeled as 'CLOSED'. Examples: desk, chair, building, door, window.
5. Substructures that represent open areas or functional spatial clusters (e.g., rooms, zones, areas, etc.) should be labeled as 'OPEN'. Examples: workspace, living area, garden, yoga zone.
6. Substructures should be presented in the order of importance according to the overall layout, with the most crucial elements prioritized at the top.
### Example-1:
Input:
{{{{
"story": "While working, he needs a desk for a drawing tablet and laptop, with enough wall and desk space to pin sketches or doodles for inspiration. Because he frequently works at night, he requires adjustable lighting that is gentle enough not to disturb sleep but bright enough for drawing. He has a large collection of books and art albums, so a dedicated bookshelf for oversized art books and a space to store art supplies or drafts is essential. Since he often stays up late sketching, he needs a comfortable bed and a quiet, relaxing rest area, with some degree of separation from the workspace. He also enjoys gaming, so the bedroom must accommodate a gaming console or display for both relaxation and creative inspiration. Occasionally, he does simple exercises like yoga or stretching, so leaving an open area is important as well.",
"aesthetic_requirements": [
"(a1) Keep balanced proportions among the main functional zones.",
"(a2) Avoid visual clutter.",
"(a3) Ensure a cohesive visual flow."
],
"manufacturing_requirements": [
"(m1) Provide adequate space based on ergonomic guidelines.",
"(m2) Ensure ventilation and natural lighting with windows or doorways.",
"(m3) Maintain safe and unobstructed passage among all areas."
],
"functional_requirements": [
"(f1) Using computer and drawing monitor.",
"(f2) Pinning or hanging sketches.",
"(f3) Rest and sleep.",
"(f4) Many large-format art books.",
"(f5) Store art supplies and drafts.",
"(f6) Gaming equipment.",
"(f7) Yoga or simple exercise.",
"(f8) Flexible or extendable layout adjustments."
]
}}}}
Output:
{{{{
"related_requirements": [
"(f1) has positive relationship with (f2), (f6), (f8)",
"(f3) has positive relationship with (a1), (a3), (m2), (m3)",
"(f4) has positive relationship with (a1), (m2)",
"(f5) has positive relationship with (a2), (m3)",
"(f6) has positive relationship with (a1), (a3), (f1)",
"(f7) has positive relationship with (a2), (m1), (m3)",
"(f8) has positive relationship with (a1), (a3), (f7)",
"(a1) has positive relationship with (a2), (a3)",
"(m1) has positive relationship with (m3)",
"(m2) has positive relationship with (m3)",
"(m3) has positive relationship with (m2)"
],
"grouped_requirements": [
{{{{
"requirement_list": ["(f1)", "(f2)", "(f6)", "(f8)"],
"call_for": "Workspace area with drawing and gaming facilities"
}}}},
{{{{
"requirement_list": ["(f3)", "(a1)", "(a2)", "(m2)", "(m3)"],
"call_for": "Rest area with separation from workspace"
}}}},
{{{{
"requirement_list": ["(f4)", "(f5)", "(a1)", "(m2)"],
"call_for": "Bookshelf and storage space for art supplies"
}}}},
{{{{
"requirement_list": ["(f7)", "(a2)", "(m1)", "(m3)"],
"call_for": "Open area for yoga and stretching"
}}}},
{{{{
"requirement_list": ["(m2)", "(m3)"],
"call_for": "Windows and door structure"
}}}}
],
"substructures": [
{{{{
"requirement_list": ["(m2)", "(m3)"],
"name": "Main door",
"type": "CLOSED",
"number_type": "SINGLE",
"number": 1,
"description": "A main door structure for entry and exit, providing easy access and ventilation.",
"reason_of_existence": "This structure connects the bedroom with the outside, ensuring safe and unobstructed passage, as well as maintaining ventilation and natural lighting.",
"is_connecting_path": "True"
}}}},
{{{{
"requirement_list": ["(m2)", "(m3)"],
"name": "Window 1",
"type": "CLOSED",
"number_type": "SINGLE",
"number": 1,
"description": "A window structure to provide natural lighting and ventilation for the workspace and rest areas.",
"reason_of_existence": "This window serves as a source of natural light and fresh air, enhancing the comfort and functionality of both the workspace and rest areas.",
"is_connecting_path": "True"
}}}},
{{{{
"requirement_list": ["(m2)", "(m3)"],
"name": "Window 2",
"type": "CLOSED",
"number_type": "SINGLE",
"number": 1,
"description": "Another window structure located in a different area of the room to ensure cross-ventilation and adequate lighting.",
"reason_of_existence": "This second window helps to create balanced lighting and airflow across the room, contributing to a comfortable and healthy environment.",
"is_connecting_path": "True"
}}}},
{{{{
"requirement_list": ["(m2)", "(m3)"],
"name": "Window 3",
"type": "CLOSED",
"number_type": "SINGLE",
"number": 1,
"description": "A third window structure placed near the rest area to enhance lighting and ventilation in that space.",
"reason_of_existence": "This window ensures the rest area is well-lit and ventilated, contributing to a comfortable sleeping environment.",
"is_connecting_path": "True"
}}}},
{{{{
"requirement_list": ["(f1)", "(f2)", "(f6)", "(f8)"],
"name": "Workspace area",
"type": "OPEN",
"number_type": "SINGLE",
"number": 1,
"description": "This area will combine a desk for computer and drawing, storage for sketches, and space for gaming equipment.",
"reason_of_existence": "This space meets the needs for working, drawing, and gaming, while allowing flexibility for different activities (adjustable layout).",
"is_connecting_path": "False"
}}}},
{{{{
"requirement_list": ["(f3)", "(a1)", "(a2)", "(m2)", "(m3)"],
"name": "Rest area",
"type": "OPEN",
"number_type": "SINGLE",
"number": 1,
"description": "A separated, quiet area for sleeping and relaxation, ensuring privacy and ergonomic space for rest.",
"reason_of_existence": "This substructure ensures a restful space, maintaining a balance in proportion, while being properly ventilated and away from the workspace.",
"is_connecting_path": "False"
}}}},
{{{{
"requirement_list": ["(f4)", "(f5)", "(a1)", "(m2)"],
"name": "Storage and bookshelf area",
"type": "OPEN",
"number_type": "SINGLE",
"number": 1,
"description": "A dedicated area to store large-format art books and art supplies, ensuring accessibility and minimizing clutter.",
"reason_of_existence": "It provides adequate storage and organizes books and supplies without cluttering the workspace.",
"is_connecting_path": "False"
}}}},
{{{{
"requirement_list": ["(f7)", "(a2)", "(m1)", "(m3)"],
"name": "Yoga/stretching area",
"type": "OPEN",
"number_type": "SINGLE",
"number": 1,
"description": "An open area left clear for performing yoga and stretching exercises, with enough space for movement.",
"reason_of_existence": "This space meets the need for occasional exercise, while ensuring unobstructed passage and proper ventilation.",
"is_connecting_path": "False"
}}}}
]
}}}}
# CRITICAL CONSTRAINTS
## Substructures in the output **MUST** be tangible objects or spaces that can be **PHYSICALLY PLACED** in the scene. **DO NOT INCLUDE ABSTRACT NOUNS.** For example, you **MUST NOT** provide substructures including "ambiance", "design", "aesthetic", "environment", "element", "theme", "measure", "setup", "atmosphere", "arrangement", "layout", "stability" and "optimization", because they are virtual and cannot be placed.
## "related_requirements", "grouped_requirements", "substructures" of the output **MUST NOT** be empty.
**FAILURE TO COMPLY WITH THESE DIRECTIVES WILL RESULT IN COMPLETE REJECTION OF YOUR OUTPUT.**
"""
)
substructure_analyzer_checker = autogen.AssistantAgent(
name="substructure_analyzer_checker",
llm_config=gpt4_json_config,
human_input_mode="NEVER",
is_termination_msg=is_termination_require,
system_message=f"""
You are an experienced 3D scene substructure analyzer checker, experting in designing scene structure and identifying the type of 3D structures.
### Task
Check the substructure list and evaluate it from the following eight aspects:
1. Non-empty related_requirements list: Is the related_requirements list non-empty? If it is empty, return false and provide your analysis; otherwise, pass.
2. Non-empty grouped_requirements list: Is the grouped_requirements list non-empty? If it is empty, return false and provide your analysis; otherwise, pass.
3. Non-empty substructure list: Is the substructure list non-empty? If it is empty, return false and provide your analysis; otherwise, pass.
4. Necessity of doors and windows: Does the scene require door and window structures? If required but not listed, return false; otherwise, pass.
5. Redundancy in the substructure list: Is the substructure list overly redundant, with many unclear substructures that can be merged or repeated, or are there substructures with inclusion relationships? If such cases exist, return false; otherwise, pass.
6. Feasibility of the substructure list: Does each substructure represent a distinct spatial division of the scene, which can be easily enclosed by a rectangular bounding box? If any substructure covers the entire scene or cannot be represented by a rectangular box, return false; otherwise, pass.
7. Balance of object quantity: Has the subdivision of substructures become overly focused on unnecessary structural divisions, neglecting the placement of actual objects? Specifically, this structure should be clearly represented by the placement of objects or a small amount of space. However, the given solution contains many redundant additional structures, which have already been considered in the previous step of the parent structure's division. If this is the case, return false, and provide your explanation and reminder. Give the advise: remove the "main area", "primary area", and some other unnecessary redundant structures.
8. Reality of substructures: Are all substructures tangible objects or spaces that can be physically placed in the scene? If there are any virtual or abstract substructures (e.g., "ambiance", "design", "aesthetic", "environment", "element", "theme", "measure", "setup", "atmosphere", "arrangement", "layout", "stability", "optimization"), give advice and return false; otherwise, pass.
### Output format
{{{{
"non-empty_related_requirements_analysis": "your analysis",
"is_non-empty_related_requirements": "True"/"False",
"non-empty_grouped_requirements_analysis": "your analysis",
"is_non-empty_grouped_requirements": "True"/"False",
"non-empty_substructure_analysis": "your analysis",
"is_non-empty_substructure": "True"/"False",
"doors_and_windows_analysis": "your analysis",
"enough_doors_and_windows": "True"/"False",
"redundant_analysis": "your analysis",
"no_redundant": "True"/"False",
"delete_redundant_substructures": ["substructure_name_1", "substructure_name_2", ...],
"feasible_analysis": "your analysis",
"is_feasible": "True"/"False",
"delete_infeasible_substructures": ["substructure_name_1", "substructure_name_2", ...],
"balance_analysis": "your analysis",
"balance_of_object_quantity": "True"/"False",
"reality_analysis": "your analysis",
"is_real": "True"/"False",
"delete_virtual_substructures": ["substructure_name_1", "substructure_name_2", ...],
"overall_decision": "True"/"False"
}}}}
"""
)
# interior_design_reviewer = autogen.AssistantAgent(
# name = "Interior_auditor",
# llm_config = gpt4_json_config,
# human_input_mode = "NEVER",
# is_termination_msg = is_termination_msg,
# system_message=f""" You are a meticulous and professional Interior Auditor.
# Your role is to critically evaluate the suggestions provided by the Interior Designer, ensuring each recommended object precisely aligns with the user's preferences, room dimensions, intended functionality, and overall design aesthetic.
# Follow the evaluation steps below and present your findings in the specified output format.
# **Evaluation Steps:**
# 1. For each suggested object, verify whether its attributes (e.g., object name, size, color, style, functionality) are consistent with the user's requirements and the room's spatial constraints.
# 2. If any object or its attributes do not align with the user’s input, room functionality, or design style, provide detailed and constructive feedback for improvements.
# 3. Evaluate whether the suggested objects form a well-integrated and functional ensemble. Pay close attention to essential pairings that naturally complement each other, such as a bed with matching nightstands, a desk with an ergonomic chair, or a vanity with an appropriate mirror.
# 4. Identify any missing or mismatched items that could disrupt the room’s balance and functionality. If combinations are incomplete or lack harmony, provide detailed recommendations for additions or modifications to create a cohesive and practical design. If certain objects appear redundant, conflict in style, or disrupt the room's flow, provide clear reasoning and suggest appropriate adjustments.
# **IMPORTANT:**
# 1. Provide objective, clear, and actionable feedback.
# 2. Ensure that recommendations maintain functionality and aesthetic balance.
# 3. Avoid suggesting any items related to doors or windows (e.g., curtains, blinds).
# 4. All evaluations should be grounded in logical reasoning and interior design principles.
# **Output Format:**
# If no issues are found, output:
# {{
# "Evaluation_Result": "SUCCESS"
# }}
# If improvements are needed, output:
# {{
# "Evaluation_Result": "REVIEW_REQUIRED",
# "Feedback": [
# {{
# "Object_Name": "<Name of the object>",
# "Identified_Issue": "<Reason for revision>",
# "Suggested_Modification": "<Recommended change>"
# }}
# ]
# }}
# """
# )
return user_proxy, requirements_analyzer, substructure_analyzer, substructure_analyzer_checker, interior_designer, designer_checker
def create_graph_agents():
user_proxy = autogen.UserProxyAgent(
name="Admin",
system_message = "A human admin.",
is_termination_msg = is_termination_msg,
code_execution_config=False
)
# - A complete list of objects and their attributes
interior_architect = autogen.AssistantAgent(
name = "Interior_architect",
llm_config = gpt4_json_config,
human_input_mode = "NEVER",
is_termination_msg = is_termination_msg,
system_message = f""" You are a highly skilled and professional Interior Architect.
In each conversation, I will provide the following information:
- Room size
- User input
- Room layout elements
- A list of previously placed objects with their placement in the room
- New objects to be placed, including their attributes
**Task**:
For each new object provided, you are required to **reason step-by-step** about its spatial position in the room. Your reasoning should:
- Identify and resolve any **conflicts** between the new object and existing objects in the scene. For example, if a table is placed to the right of a chair and a sofa is placed to the left of the table, this could result in the chair and sofa occupying the same space.
- Take into account **user input preferences** to ensure the layout aligns with their vision.
- Adhere to **basic design principles**, such as balance, scale, and proportion.
- Consider the **functionality** and **aesthetic appeal** of the room as a whole.
If the Schema_engineer reports a validation error about the JSON schema, solve the error in a way that spatially makes sense!
**Important**:
- You are not allowed to modify the positions of existing objects in the scene; only adjust the placement of new objects.
The final output must follow the JSON schema below:
{scene_graph_schema}
Note: type, description, properties, items and enum are just structural guidance and should not be output in the final result!!
## DIRECTIVES
- For the placement of each object, it should comply with basic human common sense. For example:
- The chair should face towards the table, not have its back to the table.
- The bowl cannot be placed on the wall.
- The TV MUST not be placed on the floor.
- All objects cannot be directly placed in the air.
- **Objects placed against a wall MUST face the opposite wall. A chair placed against south_wall MUST face north_wall.**
- The orientation of chairs should be determined based on their relative position to the table/desk:
- Assume the table/desk faces north_wall.
- If a chair is to the **left of** the table/desk, it should face **east_wall**.
- If a chair is to the **right of** the table/desk, it should face **west_wall**.
- If a chair is **in front** of the table/desk, it should face **south_wall**.
- If a chair is **behind** the table/desk, it should face **north_wall**.
- **A sofa MUST be placed against a wall.**
- **For the "facing" of each object, you MUST ONLY choose from ["west_wall", "east_wall", "north_wall", "south_wall"].**
- The orientation of objects should be consistent with their functional purpose and placement relative to other objects.
- **Functionally related objects MUST be placed adjacent to each other (e.g., tables and chairs, beds and bedside tables).** For example, the chair MUST be adjacent to the table/desk.
- **Tables/desks or similar objects MUST face north_wall.**
**FAILURE TO COMPLY WITH THE DIRECTIVES WILL RESULT IN COMPLETE REJECTION OF YOUR OUTPUT.**
"""
)
# schema_engineer = autogen.AssistantAgent(
# name = "Schema_engineer",
# llm_config = gpt4_json_engineer_config,
# human_input_mode = "NEVER",
# is_termination_msg = is_termination_msg,
# system_message = f""" Engineer. You listen to the output by the Interior_architect, 检查他的推理过程和是否一致(即能否根据'Placements' must logically derive from the 'Reason' process)to create a JSON file 按照预定义的模板.
# Every time when the Admin outputs objects to be in the room you will save ALL of them in the given schema!
# For the scene graph, you can use the ids for the objects that are already in the room, but only output the objects to be placed!
# If the Json_schema_debugger reports a validation error about the JSON schema, solve the error in a way that spatially makes sense!
# IMPORTANT: The inputted "Placement" key should be used for the "placement" key in the JSON object follow exatly the prepositions stated,
# do not use the information in "Facing" key for the room layout elements!!!
# IMPORTANT: For object quantities greater than one, the "placement" key gives separately the relative placement of each instance of that object in the room
# make the distinction accordingly!
# Use only the following JSON Schema to save the JSON object:
# {scene_graph_schema}
# """
# )
schema_engineer = JSONSchemaAgent(
name = "Schema_engineer",
is_termination_msg = is_termination_msg,
)
return user_proxy, interior_architect, schema_engineer
def language_summary_agents():
user_proxy = autogen.UserProxyAgent(
name="Admin",
system_message = "A human admin.",
is_termination_msg = is_termination_msg,
code_execution_config=False
)
language_architect = autogen.AssistantAgent(
name = "Language_architect",
llm_config = gpt4_md_config,
human_input_mode = "NEVER",
is_termination_msg = is_termination_msg,
system_message = f"""
You are an agent specialized in organizing and simplifying content related to the placement of objects in a room and conflict detection.
### Task:
Your task is to process the input content, which includes the placement process and conflict detection for each object in the scene, extract the key steps for arranging objects in the scene and produce a **concise and logically coherent output** in complete paragraphs.
### Input:
The input will consist of:
1. Chain of thought for requirements_analyzer, substructure_analyzer, and interior_designer.
2. Chain of thought for object placement.
3. Conflict data.
4. Scene graph.
### Output Format:
For each step, output in a smooth and logically coherent paragraph **without subheadings**. Your output should follow the provided template and include the following sections.
Present the final output in strict markdown format, following the schema below:
{{{{
## 1. Requirement Analysis
- Summarize the user's description, preferences, and requirements in a complete paragraph.
- Highlight key elements such as room dimensions, style preferences, and functional needs.
## 2. Area Decomposition
- Describe how the scene is decomposed into substructures based on user requirements in a complete paragraph.
- List each substructure and its purpose.
## 3. Object Recommendations
- Detail the objects recommended for each substructure in a complete paragraph.
- Include object attributes such as style, dimensions, and functionality.
## 4. Scene Graph
- For each object, describe the following in a complete paragraph:
- **Placement Reason**: Why the object is necessary based on user requirements and substructure analysis.
- **Placement**: Specific location in the room (e.g., center, against a wall).
- **Dimensions**: Object size.
- **Direction**: Orientation of the object.
- **Placement Process**: Detailed reasoning and steps taken to place the object, including how it aligns with user preferences and design principles. Include consideration of existing objects in the room and their relative positional relationships.
## 5. Global Check
- **Conflict Description**: Conflict includes layout conflict and size conflict. Identify the conflict and the objects involved in a complete paragraph.
- **Conflict Resolution**: Explain how the conflict was resolved, including the repositioning of objects and the deletion of objects.
}}}}
### Instructions:
- Extract key information from the input and organize it logically.
- Remove redundancy while retaining all critical details.
- Use clear and concise language, avoiding technical jargon where possible.
- Ensure the output flows naturally and is easy to understand.
- Maintain logical connections between sections using appropriate transitions.
- In the **Scene Graph** section, **Dimensions** of **every object**, namely its length, width, and height, **MUST** be accurately described.
- In the **Scene Graph** section, the descriptions of **Placement Reason** and **Placement Process** for each object should be detailed and comprehensive, clearly reflecting the complete thought process. This includes explaining **why the object is placed in a specific location** and considering its relationship to existing objects in the room. Additionally, the descriptions should align as closely as possible with the **<REASON>** section of the provided **Chain of Thought for Object Placement**.
### Example:
{{{{
## 1. Requirement Analysis
Based on the user's description, the primary goal is to create a modern-style game room centered around a pool table. The room dimensions are specified as 4.0 meters by 4.0 meters with a ceiling height of 2.5 meters. The user explicitly requested a pool table and emphasized a preference for a modern aesthetic.
## 2. Area Decomposition
The first substructure is the Pool Table Area, designated as the central zone for the pool table. Next is the Storage Area, intended for storing billiard accessories. The Lighting Area focuses on ambient lighting to enhance the modern aesthetic, while the Leisure Area is designed for social interaction and relaxation.
## 3. Object Recommendations
For the Pool Table Area, a modern-style pool table with dimensions of 3.444 meters by 1.819 meters by 0.801 meters is recommended. In the Storage Area, a modern metal shelving unit measuring 1.5 meters by 0.3 meters by 2.0 meters is proposed to store billiard accessories. The Lighting Area features an LED light strip with a length of 4.0 meters, installed along the ceiling perimeter. For the Leisure Area, a modern-style carpet (4.0 meters by 4.0 meters), a sofa (2.0 meters by 0.8 meters), and a modern glass coffee table (1.0 meters by 0.6 meters) are recommended to create a comfortable and stylish space.
## 4. Scene Graph
Starting with the pool table, the user explicitly requested it as the central element. Given the room's dimensions (4.0m x 4.0m x 2.5m) and the pool table's size (3.444m x 1.819m x 0.801m), it is placed centrally to provide adequate space on all sides for movement and gameplay. This central placement ensures the pool table remains the focal point, adhering to modern design principles while maintaining balance and proportion. The pool table is oriented parallel to the walls, ensuring optimal gameplay and visual harmony with the room's layout.
Next, the shelving unit is positioned against the north wall to the right of the pool table. This placement ensures easy access for storing billiard accessories while avoiding spatial conflicts with the pool table. The shelving unit's dimensions (1.5m x 0.3m x 2.0m) comfortably fit against the wall, maintaining the room's modern aesthetic and functional requirements. Its placement against the north wall complements the pool table without overwhelming the space, ensuring the room remains uncluttered and visually balanced.
The LED light strip is installed along the ceiling perimeter to enhance ambient lighting. This placement ensures the light strip does not interfere with the pool table or shelving unit while providing a modern lighting solution. The 4.0-meter length of the LED strip allows it to extend across the ceiling perimeter, creating a cohesive and visually appealing lighting scheme that complements the room's modern style.
For the carpet, it is placed on the floor, covering the entire room. The carpet's dimensions (4.0m x 4.0m) allow it to extend under the pool table, defining the leisure area and enhancing the room's aesthetic. Its thin profile ensures it does not interfere with the pool table's functionality or the shelving unit's placement. The carpet's neutral color and modern style align with the user's preferences, creating a cohesive visual boundary for the game area.
The sofa is positioned along the south wall, facing the pool table. This placement ensures the sofa complements the leisure aspect of the game room without disrupting the room's flow. The sofa's dimensions (2.0m x 0.8m) allow it to fit comfortably against the south wall, maintaining balance and proportion. Its orientation toward the pool table enhances the room's functionality as a social and relaxation space while adhering to modern design principles.
Finally, the coffee table is placed in front of the sofa, facing the north wall. This placement ensures it is adjacent to the sofa, providing a functional surface for drinks and items while maintaining the room's modern aesthetic. The coffee table's dimensions (1.0m x 0.6m) and transparent material ensure it does not visually clutter the space, preserving an open feel. Its position on the carpet, in the middle of the room, creates a cohesive leisure area that enhances the room's usability and visual appeal.
## 5. Global Check
During the placement process, a conflict was identified: the central area of the room was too small to accommodate all objects, including the carpet, pool table, shelving unit, sofa, and coffee table. To resolve this, the shelving unit was removed based on its lower functional priority compared to the pool table and leisure area.
}}}}
"""
)
return user_proxy, language_architect
def calculation_summary_agents():
user_proxy = autogen.UserProxyAgent(
name="Admin",
system_message = "A human admin.",
is_termination_msg = is_termination_msg,
code_execution_config=False
)
calculation_architect = autogen.AssistantAgent(
name = "Calculation_architect",
llm_config = gpt4_md_config,
human_input_mode = "NEVER",
is_termination_msg = is_termination_msg,
system_message = f"""
You are an agent specialized in organizing and simplifying detailed calculation processes related to object placement in a room.
### Task:
Process the input, which consists of verbose and redundant calculation logs, and produce a concise output that clearly presents the calculation process for each object in the order they were placed.
### Input:
The input will be a detailed log of calculations for placing objects in a room, including constraint handling, position calculations, boundary adjustments, and collision checks.
The log contains repetitive and redundant information, such as multiple checks for the same constraints and verbose descriptions of calculations.
### Output Requirements:
- **Object Placement Order**: Present the calculation process in the sequence objects were placed, maintaining parent-child relationships.
- **Calculation Steps** (per object):
- **Format**
- **Reason**: Concise explanation of the calculation purpose (e.g., constraint application, collision check).
- **Calculation Process**:
- Key formulas with object dimensions, constraint parameters, and intermediate results.
- Parent-child spatial relationships (e.g., 'parent_id' for nested objects).
- Directional constraints (e.g., rotation differences, prepositions like "behind", "in front").
- **Conclusion**: Final result of the calculation and how it affects the object's placement (e.g., final coordinates and constraint compliance status).
- **Content**
- **Spatial Relationships**:
- **Rotation Difference**: '|θ_parent - θ_child|' calculation with explicit object IDs.
- **Directional Constraints**: Preposition logic (e.g., "behind", "in front") with dimension selection (length / width).
- **Constraint Propagation**:
- **Size Constraint**: 'max(0.0, child_size + cluster_size)' for directional constraints.
- **Cluster Size**: Aggregated values from child objects (e.g., 'left of', 'in front').
- **Position Calculation**:
- **Boundary Adjustment**: Formulaic derivation of 'x_min', 'x_max', etc., referencing parent positions.
- **Collision Check**: Overlap detection logic (e.g., 'parent_x ± parent_length / 2' vs. 'child_x ± child_length / 2').
- **Final Placement**:
- **Coordinates**: Final position with cluster constraint compliance.
- **Status**: Collision outcome (e.g., "No collision detected").
### Instructions:
1. **Extract Key Information**:
- Identify essential steps (e.g., constraint resolution, boundary adjustments).
- Extract exact dimensions and constraint parameters from input logs.
- Track parent-child hierarchy and spatial relationships.
2. **Remove Redundancy**:
- Eliminate duplicate constraint checks for the same object.
- Condense repetitive collision detection logic.
3. **Organize by Placement Order**:
- Follow the order of object placement (e.g., 'pool_table_1' first, then its children).
- Nest child objects under their parent in the output.
4. **Include All Formulas**:
- Retain rotation difference calculations ('|θ₁ - θ₂|').
- Show cluster size aggregation (e.g., 'max(0.0, child_size + cluster_size)').
- Include boundary adjustment formulas (e.g., 'x_min = parent_x - parent_length / 2 - child_length / 2').
- Explain their purpose and outcome.
5. **Clarify and Simplify**:
- Replace ambiguous terms with specific object IDs (e.g., "sofa_1" instead of "child object").
- Use consistent notation for coordinate systems (x, y, z).
- Explicitly state collision outcomes (e.g., "No collision detected").
### Instructions for Calculation Steps:
1. **Constraint Calculation**:
- For each directional preposition (e.g., "behind", "in front"):
- Compute rotation difference between parent and child.
- Select primary dimension (length / width) based on rotation alignment.
- Calculate constraint as 'max(0.0, child_size + child_cluster_size)'.
- For non-directional constraints (e.g., "under"):
- Skip size constraint but apply spatial boundaries.
2. **Cluster Size Aggregation**:
- Sum child sizes and their cluster constraints along the selected dimension.
- Example: 'sofa_1 cluster size (in front) = coffee_table_1.size (1.0) + coffee_table_1.cluster_size (0.0) = 1.0'.
3. **Position Calculation**:
- For wall / ceiling constraints:
- Use formula: 'x_min = wall_x - wall_length / 2 + object_length / 2'.
- For relative constraints (e.g., "under pool_table_1"):
- Formula: 'x_min = parent_x - parent_length / 2 - object_length / 2'.
4. **Collision Detection**:
- Check overlap using 'parent_x ± parent_length / 2' vs. 'child_x ± child_length / 2'.
- Explicitly state collision outcomes (e.g., "Overlap detected: 1.722 ≤ 2.4041 ≤ 2.278 → No collision").
### Output Format:
Present the final output in strict markdown format, following the schema below:
{{{{
For object: string (required)
- parent object: string (optional)
- *description*: Identifier for the parent object (if applicable)
- calculation_steps: array of calculation steps (required)
1. reason: Calculate rotation difference with {{{{child_id}}}}
- calculation: array of strings
- *description*: Detailed calculation steps
- conclusion: string
- *description*: Summary of the calculation result
2. reason: Calculate size constraint for {{{{preposition}}}} relation
- calculation: array of strings
- *description*: Detailed calculation steps
- conclusion: string
- *description*: Summary of the calculation result
3. reason: Calculate possible positions based on {{{{constraint_name}}}} constraint
- calculation: array of strings
- *description*: Detailed calculation steps
- conclusion: string
- *description*: Summary of the calculation result
4. reason: Adjust boundaries for valid placement
- calculation: array of strings
- *description*: Detailed calculation steps
- conclusion: string
- *description*: Summary of the calculation result
5. reason: Collision check with {{{{object_id}}}}
- calculation: array of strings
- *description*: Detailed calculation steps
- conclusion: string
- *description*: Summary of the calculation result
6. reason: Final position calculation
- calculation: array of strings
- *description*: Detailed calculation steps
- conclusion: string
- *description*: Summary of the final position
}}}}
### Example:
{{{{
For pool_table_1
- calculation_steps:
1. reason: Calculate rotation difference with sofa_1
- calculation:
- Rotation of pool_table_1: 0.0°
- Rotation of sofa_1: 0.0°
- Rotation difference: |0.0 - 0.0| = 0.0°
- conclusion: Using length dimension for directional constraint
2. reason: Calculate size constraint based on 'behind' relation
- calculation:
- sofa_1 size: 2.0 (length)
- coffee_table_1 cluster size (in front): 1.0
- Total constraint: 2.0 (sofa_1 length) + 0.0 (non-adjacent) = 2.0
- conclusion: Cluster constraint (y_neg): 2.0
3. reason: Calculate possible positions based on 'middle of the room' constraint
- calculation:
- pool_table_1 size: length=3.444, width=1.819, height=0.801
- Room size: 4.0x4.0x2.5
- x_min = 2.0 - 4.0/2 + 3.444/2 = 1.722
- x_max = 2.0 + 4.0/2 - 3.444/2 = 2.278
- y_min = 2.0 - 4.0/2 + 1.819/2 = 0.9095
- y_max = 2.0 + 4.0/2 - 1.819/2 = 3.0905
- z_min = z_max = 0.801/2 = 0.4005
- conclusion: Possible position: (1.722, 2.278, 0.9095, 3.0905, 0.4005, 0.4005)
4. reason: Final position calculation
- calculation:
- Adjusted cluster constraint: x(1.722-2.278), y(0.9095-3.0905)
- Final coordinates: x=2.0695, y=3.0301, z=0.4005
- conclusion: Final position: x: 2.0695, y: 3.0301, z: 0.4005
For sofa_1
- parent object: pool_table_1
- calculation_steps:
1. reason: Calculate rotation difference with coffee_table_1
- calculation:
- Rotation of sofa_1: 0.0°
- Rotation of coffee_table_1: 0.0°
- Rotation difference: |0.0 - 0.0| = 0.0°
- conclusion: Using length dimension for directional constraint
2. reason: Calculate size constraint based on 'in front' relation
- calculation:
- coffee_table_1 size: 1.0 (length)
- Cluster size (in front): max(0.0, 1.0) = 1.0
- conclusion: sofa_1 cluster size (in front): 1.0
3. reason: Calculate possible positions based on 'south_wall' constraint
- calculation:
- sofa_1 size: length=2.0, width=0.8, height=0.75
- x_min = 2.0 - 4.0/2 + 2.0/2 = 1.0
- x_max = 2.0 + 4.0/2 - 2.0/2 = 3.0
- y_min = y_max = 0.4
- z_min = z_max = 0.375
- conclusion: Possible position: (1.0, 3.0, 0.4, 0.4, 0.375, 0.375)
4. reason: Adjust for 'behind pool_table_1' constraint
- calculation:
- y_min = max(0.4, 3.0301 - 1.819/2 - 0.8/2) = 1.7206
- y_max = min(0.4, 3.0301 + 1.819/2 + 0.8/2) = 0.4
- conclusion: Final position: x: 2.3208, y: 0.4, z: 0.375
For coffee_table_1
- parent object: sofa_1
- calculation_steps:
1. reason: Calculate size constraint for 'in front' relation
- calculation:
- coffee_table_1 size: 1.0 (length)
- Cluster size (in front): max(0.0, 1.0) = 1.0
- conclusion: sofa_1 cluster size (in front): 1.0
2. reason: Calculate possible positions based on 'in front of sofa_1' constraint
- calculation:
- x_min = 2.3208 - 2.0/2 + 1.0/2 = 1.8208
- x_max = 2.3208 + 2.0/2 - 1.0/2 = 2.8208
- y_min = y_max = 0.4 + 0.8/2 + 0.6/2 = 1.1
- z_min = z_max = 0.2
- conclusion: Possible position: (1.8208, 2.8208, 1.1, 1.1, 0.2, 0.2)
3. reason: Collision check with pool_table_1
- calculation:
- Overlap detection: 1.722 ≤ 2.4041 ≤ 2.278 → No collision
- Final coordinates: x=2.4041, y=1.1, z=0.2
- conclusion: Final position: x: 2.4041, y: 1.1, z: 0.2
For carpet_1
- parent object: pool_table_1
- calculation_steps:
1. reason: Calculate size constraint for 'under' relation
- calculation:
- carpet_1 size: 4.0x4.0x0.01
- Cluster size (under): 0.0 (non-directional)
- conclusion: No directional constraint applied
2. reason: Calculate possible positions based on 'middle of the room' constraint
- calculation:
- x_min = x_max = 2.0
- y_min = y_max = 2.0
- z_min = z_max = 0.005
- conclusion: Possible position: (2.0, 2.0, 2.0, 2.0, 0.005, 0.005)
3. reason: Adjust for 'under pool_table_1' constraint
- calculation:
- x_min = max(2.0, 2.0695 - 3.444/2 - 4.0/2) = 2.0
- y_min = max(2.0, 3.0301 - 1.819/2 - 4.0/2) = 2.0
- conclusion: Final position: x: 2.0, y: 2.0, z: 0.005
}}}}
# DIRECTIVES
Carefully analyze the input. For each object, the "calculation_steps" **MUST** include the following six reasons in order:
1. Calculate rotation difference with {{{{child_id}}}}
2. Calculate size constraint for {{{{preposition}}}} relation
3. Calculate possible positions based on {{{{constraint_name}}}} constraint
4. Adjust boundaries for valid placement
5. Collision check with {{{{object_id}}}}
6. Final position calculation
**The output MUST be in Markdown format. MUST NOT use JSON format. MUST NOT use any curly braces, square brackets, double quotes.**
**If the output is in JSON format, you MUST convert it to Markdown format.**
**FAILURE TO COMPLY WITH THE DIRECTIVES WILL RESULT IN COMPLETE REJECTION OF YOUR OUTPUT.**
"""
)
return user_proxy, calculation_architect
# interior_architecture_reviewer = autogen.AssistantAgent(
# name = "Interior_architect_reviewer",
# llm_config = gpt4_json_config,
# human_input_mode = "NEVER",
# is_termination_msg = is_termination_msg,
# system_message=f""" You are a meticulous and professional Interior Architect Reviewer.
# Your role is to critically evaluate the suggestions provided by the Interior Designer, ensuring each recommended object precisely aligns with the user's preferences, room dimensions, intended functionality, and overall design aesthetic.
# Follow the evaluation steps below and present your findings in the specified output format.
# **Evaluation Steps:**
# 1. For each suggested object, verify whether its attributes (e.g., object name, size, color, style, functionality) are consistent with the user's requirements and the room's spatial constraints.
# 2. If any object or its attributes do not align with the user’s input, room functionality, or design style, provide detailed and constructive feedback for improvements.
# 3. Evaluate whether the suggested objects form a well-integrated and functional ensemble. Pay close attention to essential pairings that naturally complement each other, such as a bed with matching nightstands, a desk with an ergonomic chair, or a vanity with an appropriate mirror.
# 4. Identify any missing or mismatched items that could disrupt the room’s balance and functionality. If combinations are incomplete or lack harmony, provide detailed recommendations for additions or modifications to create a cohesive and practical design. If certain objects appear redundant, conflict in style, or disrupt the room's flow, provide clear reasoning and suggest appropriate adjustments.
# **IMPORTANT:**
# 1. Provide objective, clear, and actionable feedback.
# 2. Ensure that recommendations maintain functionality and aesthetic balance.
# 3. Avoid suggesting any items related to doors or windows (e.g., curtains, blinds).
# 4. All evaluations should be grounded in logical reasoning and interior design principles.
# **Output Format:**
# If no issues are found, output:
# {{
# "Evaluation_Result": "SUCCESS"
# }}
# If improvements are needed, output:
# {{
# "Evaluation_Result": "REVIEW_REQUIRED",
# "Feedback": [
# {{
# "Object_Name": "<Name of the object>",
# "Identified_Issue": "<Reason for revision>",
# "Suggested_Modification": "<Recommended change>"
# }}
# ]
# }}
# """
# )
# Begin by analyzing the objects suggested by the Interior Designer and classify them into hierarchical Parent-Child trees based on their function and architectural logic. For each object, decide whether its Parent Object or a Child Object.
# - Parent: Think about and Specify the object's parent in one list item. If there is no parent, set it to None.
# - Children: Think about and List any child objects associated with this object in one list item. If there is no children, set it to None.
# - Object Name (e.g., bed, desk, chair, monitor, bookshelf)
# - Parent Object: Larger or central items that define functional zones or support Child Objects (e.g., bed, desk, sofa).
# - Child Object: Objects that complement or rely on Parent Objects (e.g., nightstand beside a bed, lamp on a desk).
# - The Parent-Child relationship is relative. For example:
# - A bed is the Parent of a nightstand, and the nightstand is the Parent of a lamp.
# - Parent objects can be suggested by the Interior Designer or be part of room layout elements: ['south_wall', 'north_wall', 'west_wall', 'east_wall', 'middle of the room', 'ceiling'].
# - Child objects must be suggested by the Interior Designer.
# **Important:**
# If the Interior_architect_reviewer provides feedback, thoroughly review its comments and, if necessary, revise your suggestions by adjusting the 对应 object's Placement, Proximity, or Facing to better align with the 用户的输入 design principles.如果你觉得不需要修改,请提供你的理由,并且不做修改
# interior_architect = autogen.AssistantAgent(
# name = "Interior_architect",
# llm_config = gpt4_json_config,
# human_input_mode = "NEVER",
# is_termination_msg = is_termination_msg,
# system_message = f""" You are a highly skilled and professional Interior Architect.
# Your role is to analyze the user input, step-by-step think about where the optimal placement for each object suggested by the Interior Designer.
# Each reasoning step and the final output must be logically sound and well-justified.
# **Important**:
# If the quantity of an object is greater than one, you have to find a place for each instance of this object separately! But give all this information in **one list** item!
# This means the output should have one dictionary for each object, but all their instances (quantity higher than one) should be in the same dictionary!
# **Hint:**
# 1. Begin by analyzing the user's input and the objects suggested by the Interior Designer. Identify the main objects (typically objects that are most relevant to the user's input, define a specific area's function, are larger in size, or support other objects) in the scene prioritize their placement.
# 2. Based on common sense and design principles, **step-by-step reason** the placement, proximity, and facing direction of room layout elements or these main objects.
# 3. Spatial conflicts must be carefully avoided. For example, if a table is placed to the right of a chair and a sofa is placed to the left of the table, this could result in the chair and sofa occupying the same space.
# 4. Provide explicit answers for EACH object on the following four aspects:
# - Object Name: The same as in the suggested list.
# - Placement:
# Think about the placement of this object based on the following rules:
# - Identify a relative place in the whole room (e.g., on the middle of the floor, in the north-west corner, on the east wall, right of the desk, on the bookshelf...).
# - For relative placement with the room layout elements (walls, the middle of the room, ceiling) use the prepositions "on", "in the corner". You are not allowed to use any prepositions different from the ones above!!
# - For relative placement with other objects in the room use the prepositions "on", "left of", "right of", "in front", "behind", "under". You are not allowed to use any prepositions different from the ones above!!
# - Explicitly state the placement for each instance (e.g., one is on the south_wall, one is left of desk_1).
# - Provide detailed and structured descriptions for each object's placement.
# - Proximity:
# Think about the proximity of this object to the relative placement objects. If there are multiple relative placement objects, prioritize the first one listed for proximity calculations.
# - Adjacent: The object is physically contacting the other object or it is supported by the other object or they are touching or they are close to each other.
# - Not Adjacent: The object is not physically contacting the other object or it is distant from the other object.
# - Facing:
# - Think about which wall (west/east/north/south_wall/floor/ceiling) this object should be facing and explicitly state this (ex. one is facing the south_wall)!
# Follow the JSON schema below:
# {interior_architect_schema}
# The 'Reason' array should present a detailed, step-by-step explanation of the thought process used to determine the placement, proximity, and facing direction of each object, following a clear and logical chain-of-thought reasoning structure. Conclude your reasoning with specific and actionable placement recommendations for each object.
# The 'Placements' must logically derive from the 'Reason' process and remain consistent with the provided placement recommendations.
# """
# )
|